Regularizers¶
- class mumott.optimization.regularizers.Laplacian[source]¶
Regularizes using the Laplacian of the coefficients. Suitable for orthonormal representations, e.g., spherical harmonics.
- get_regularization_norm(coefficients, get_gradient=False)[source]¶
Retrieves the regularization norm and possibly the gradient based on the provided coefficients. The norm is the 2-norm of the discrete nearest-neighbour Laplacian.
This is in effect a smoothing kernel that enforces continuity between the tensors of neighbouring voxels. The calculation is most suitable for orthonormal representations. If the representation is in spherical harmonics, the norm corresponds to maximimzing the covariance between neighbours.
- Parameters
coefficients (
ndarray
[Any
,dtype
[float
]]) – Annp.ndarray
of values, with shape(X, Y, Z, W)
, where the last channel contains, e.g., tensor components.get_gradient (
bool
) – IfTrue
, returns a'gradient'
of the same shape ascoefficients
. Otherwise, the entry'gradient'
will beNone
. Defaults toFalse
.
- Return type
- Returns
A dictionary with two entries,
regularization_norm
andgradient
.
- class mumott.optimization.regularizers.TotalVariation(delta=0.01)[source]¶
Regularizes using the symmetric total variation, i.e., the root-mean-square difference between nearest neighbours. It is combined with a Huber norm, using the squared differences at small values, in order to improve convergence. Suitable for scalar fields or tensor fields in local representations. Tends to reduce noise.
In two dimensions, the total variation spliced with its squared function like a Huber loss can be written
\[\begin{split}\mathrm{TV}_1(f(x, y)) = \frac{1}{h}\sum_i ((f(x_i, y_i) - f(x_i + h, y_i))^2 + (f(x_i, y_i) - f(x_i - h, y_i))^2 + \\ (f(x_i, y_i) - f(x_i, y_i + h))^2 + (f(x_i, y_i) - f(x_i, y_i - h))^2))^{\frac{1}{2}} - 0.5 \delta\end{split}\]If \(\mathrm{TV}_1 < 0.5 \delta\) we instead use
\[\begin{split}\mathrm{TV}_2(f(x, y)) = \frac{1}{2 \delta h^2}\sum_i (f(x_i, y_i) - f(x_i + h, y_i))^2 + (f(x_i, y_i) - f(x_i - h, y_i))^2 + \\ (f(x_i, y_i) - f(x_i, y_i + h))^2 + (f(x_i, y_i) - f(x_i, y_i - h))^2\end{split}\]See also the Wikipedia articles on total variation denoising and Huber loss
- Parameters
delta (float) – Below this value, the scaled square of the total variation is used as the norm. This makes the norm differentiable everywhere, and can improve convergence. If :attr`delta` is
None
, the standard total variation will be used everywhere, and the gradient will be0
at the singular point where the norm is0
.
- get_regularization_norm(coefficients, get_gradient=False)[source]¶
Retrieves the isotropic total variation, i.e., the symmetric root-mean-square difference between nearest neighbours.
- Parameters
coefficients (
ndarray
[Any
,dtype
[float
]]) – Annp.ndarray
of values, with shape(X, Y, Z, W)
, where the last channel contains, e.g., tensor components.get_gradient (
bool
) – IfTrue
, returns a'gradient'
of the same shape ascoefficients
. Otherwise, the entry'gradient'
will beNone
. Defaults toFalse
.
- Return type
- Returns
A dictionary with two entries,
regularization_norm
andgradient
.
- class mumott.optimization.regularizers.L1Norm[source]¶
Regularizes using the \(L_1\) norm of the coefficient vector, also known as the Manhattan or taxicab norm. Suitable for scalar fields or tensor fields in local representations. Tends to reduce noise.
The \(L_1\) norm of a vector \(x\) is given by \(\sum{\vert x \vert}\).
See also this Wikipedia article.
- get_regularization_norm(coefficients, get_gradient=False)[source]¶
Retrieves the \(L_1\) norm, also called the Manhattan or taxicab norm, of the coefficients. Appropriate for use with scalar fields or tensor fields in local basis sets.
- Parameters
coefficients (
ndarray
[Any
,dtype
[float
]]) – Annp.ndarray
of values, with shape(X, Y, Z, W)
, where the last channel contains, e.g., tensor components.get_gradient (
bool
) – IfTrue
, returns a'gradient'
of the same shape ascoefficients
. Otherwise the entry'gradient'
will beNone
. Defaults toFalse
.
- Return type
- Returns
A dictionary with two entries,
regularization_norm
andgradient
.
- class mumott.optimization.regularizers.L2Norm[source]¶
Regularizes using the \(L_2\) norm of the coefficient vector, also known as the Euclidean norm. Suitable for most representations, including non-local ones. Tends to reduce large values, and often leads to fast convergence.
The \(L_2\) norm of a vector \(x\) is given by \(\sum{\vert x \vert^2}\).
See also the Wikipedia article on the Euclidean norm
- get_regularization_norm(coefficients, get_gradient=False)[source]¶
Retrieves the \(L_2\) norm, of the coefficient vector. Appropriate for use with scalar coefficients or local basis sets.
- Parameters
coefficients (
ndarray
[Any
,dtype
[float
]]) – Annp.ndarray
of values, with shape(X, Y, Z, W)
, where the last channel contains, e.g., tensor components.get_gradient (
bool
) – IfTrue
, returns a'gradient'
of the same shape ascoefficients
. Otherwise, the entry'gradient'
will beNone
. Defaults toFalse
.
- Return type
- Returns
A dictionary with two entries,
regularization_norm
andgradient
.
- class mumott.optimization.regularizers.HuberNorm(delta=1.0)[source]¶
Regularizes using the Huber norm of the coefficient, which splices the :math`L_1` and \(L_2\) norms. Suitable for scalar fields or tensor fields in local representations. Tends to reduce noise while converging more easily than the \(L_1\) loss function.
The Huber norm of a vector \(x\) is given by \(R(\vec{x}) = \sum_i L(x_i)\), where \(L(x_i)\) is given by
\[\begin{split}L(x_i) = \begin{Bmatrix}\vert x_i \vert - 0.5 \delta & \quad \text{if } \vert x_i \vert > \delta \\ \dfrac{x^2}{2 \delta} & \quad \text{if } \vert x_i \vert \leq \delta\end{Bmatrix}\end{split}\]See also the Wikipedia article on the Huber loss.
- Parameters
delta (float) – The threshold value for the Huber norm. Must be greater than
0
. Default value is1.
, but the appropriate value is data-dependent.
- get_regularization_norm(coefficients, get_gradient=False)[source]¶
Retrieves the Huber loss of the coefficients. Appropriate for use with scalar fields or tensor fields in local representations.
- Parameters
coefficients (
ndarray
[Any
,dtype
[float
]]) – Annp.ndarray
of values, with shape(X, Y, Z, W)
, where the last channel contains, e.g., tensor components.get_gradient (
bool
) – IfTrue
, returns a'gradient'
of the same shape ascoefficients
. Otherwise, the entry'gradient'
will beNone
. Defaults toFalse
.
- Return type
- Returns
A dictionary with two entries,
regularization_norm
andgradient
.