yamle.regularizers.gradient module

class yamle.regularizers.gradient.GradientNoiseRegularizer(eta, gamma, *args, **kwargs)

Bases: BaseRegularizer

This is a class for a gradient noise regularization.

It adds a noise sampled from a normal distribution with mean 0 and standard deviation std to the gradient.

It follows the paper: https://arxiv.org/pdf/1511.06807.pdf

Parameters:

• eta (float) – The standard deviation of the normal distribution from which the noise is sampled.

• gamma (float) – The factor by which the noise is multiplied.

on_after_backward(model, epoch, *args, **kwargs)

Add noise to the gradients after the backward pass.

Return type:

None

static add_specific_args(parser)

This method is used to add specific arguments to the parser.

Return type:

ArgumentParser