yamle.regularizers.model module#
- class yamle.regularizers.model.ShrinkAndPerturbRegularizer(l, std, start_epoch, end_epoch, epoch_frequency, *args, **kwargs)[source]#
Bases:
BaseRegularizerThis is a class for a shrink and perturb regularization.
It shrinks the weights by a factor of l and adds a noise sampled from a normal distribution with mean 0 and standard deviation std to the weights at a certain epoch frequency.
There is also a second argument which limits the starting epoch and the ending epoch within which the shrink and perturb regularization is applied.
It follows the paper: https://arxiv.org/pdf/1910.08475.pdf
- Parameters:
l¶ (float) – The factor by which the weights are shrunk.
std¶ (float) – The standard deviation of the normal distribution from which the noise is sampled.
start_epoch¶ (int) – The epoch at which the shrink and perturb regularization starts. Default is 0, which means that the regularization is applied from the beginning of the training.
end_epoch¶ (int) – The epoch at which the shrink and perturb regularization ends. Default is -1, which means that the regularization is applied until the end of the training.
epoch_frequency¶ (int) – The frequency at which the shrink and perturb regularization is applied.
- on_after_train_epoch(model, epoch, *args, **kwargs)[source]#
Add noise to the weights after a given training epoch.
For all parameters that require gradients, the weights are shrunk by a factor of l and a noise sampled from a normal distribution with mean 0 and standard deviation std is added to the weights.
- Return type:
None