yamle.regularizers.feature module

class yamle.regularizers.feature.L1FeatureRegularizer

Bases: BaseRegularizer

This is a class for L1 regularization for the output features.

class yamle.regularizers.feature.L2FeatureRegularizer

Bases: BaseRegularizer

This is a class for L2 regularization for the output features.

class yamle.regularizers.feature.InnerProductFeatureRegularizer(dim=1, *args, **kwargs)

Bases: BaseRegularizer

This is a class for inner product regularization.

Given a tensor x which can be split in dimension dim into n tensors x_1, …, x_n, the regularization loss is calculated as:

loss = sum_{i=1}^{n} sum_{j=i+1}^{n} x_i * x_j loss = loss / (n*(n-1)/2)

Parameters:

dim (int) – The dimension over which split the tensor to then calculate the inner product as a cartesian product.

static add_specific_args(parser)

This method is used to add specific arguments to the parser.

Return type:

ArgumentParser

class yamle.regularizers.feature.CosineSimilarityFeatureRegularizer(dim=1, *args, **kwargs)

Bases: InnerProductFeatureRegularizer

This is a class for cosine similarity regularization.

Given a tensor x which can be split in dimension dim into n tensors x_1, …, x_n, the regularization loss is calculated as:

loss = sum_{i=1}^{n} sum_{j=i+1}^{n} cos(x_i, x_j) loss = loss / (n*(n-1)/2)

The cos function is the cosine similarity between x_i and x_j. cos(x_i, x_j) = x_i * x_j / (||x_i|| * ||x_j||)

class yamle.regularizers.feature.CorrelationFeatureRegularizer(dim=1, *args, **kwargs)

Bases: CosineSimilarityFeatureRegularizer

This is a class for correlation regularization.

Correlation is the cosine similarity between centered versions of x and y. Unlike the cosine, the correlation is invariant to both scale and location changes of x and y.