Optimizers¶
- class mumott.optimization.optimizers.LBFGS(loss_function, **kwargs)[source]¶
This Optimizer makes the LBFGS algorithm from scipy.optimize available for usage with a
LossFunction
.- Parameters
loss_function (LossFunction) – The loss function to be minimized using this algorithm.
kwargs (Dict[str, Any]) – Miscellaneous options. See notes for valid entries.
Notes
Valid entries in
kwargs
are- x0
Initial guess for solution vector. Must be the same size as
residual_calculator.coefficients
. Defaults toloss_function.initial_values
.- bounds
Used to set the
bounds
of the optimization method, see scipy.optimize.minimize documentation for details. Defaults toNone
.- maxiter
Maximum number of iterations. Defaults to
10
.- disp
Whether to display output from the optimizer. Defaults to
False
- maxcor
Maximum number of Hessian corrections to the Jacobian. Defaults to
10
.- iprint
If
disp
is true, controls output with no output ifiprint < 0
, convergence output only ifiprint == 0
, iteration-wise output if0 < iprint <= 99
, and sub-iteration output ifiprint > 99
.- maxfun
Maximum number of function evaluations, including line search evaluations. Defaults to
20
.- ftol
Relative change tolerance for objective function. Changes to absolute change tolerance if objective function is
< 1
, see scipy.optimize.minimize documentation, which may lead to excessively fast convergence. Defaults to1e-3
.- gtol
Convergence tolerance for gradient. Defaults to
1e-5
.
- property no_tqdm¶
Whether to avoid making a
tqdm
progress bar.
- optimize()[source]¶
Executes the optimization using the options stored in this class instance. The optimization will continue until convergence, or until the maximum number of iterations (
maxiter
) is exceeded.- Return type
- Returns
A
dict
of optimization results. See scipy.optimize.OptimizeResult for details. The entry'x'
, which contains the result, will be reshaped using the shape of the gradient fromloss_function
.
- class mumott.optimization.optimizers.GradientDescent(loss_function, **kwargs)[source]¶
This Optimizer is a gradient descent (sometimes called steepest descent) solver, which can be set to terminate based on the loss function and/or the maximum number of iterations.
It also supports the use of Nestorov accelerated momentum, which features a look-ahead momentum term based on the gradient of the previous iterations.
The update sequence may be written
\[\begin{split}x &\leftarrow x - (p + \alpha(\nabla(x - p) + \Lambda)) \\ p &\leftarrow \beta(p + \alpha(\nabla(x - p) + \Lambda))\end{split}\]where \(x\) are the optimization coefficients, \(p\) is the momentum, and \(\Lambda\) is the regularization term. \(\alpha\) is the step size and \(\beta\) is the Nestorov momentum weight.
- Parameters
loss_function (LossFunction) – The loss function to be minimized using this algorithm.
kwargs (Dict[str, Any]) – Miscellaneous options. See notes for valid entries.
Notes
- Valid entries in
kwargs
are - x0
Initial guess for solution vector. Must be the same size as
residual_calculator.coefficients
. Defaults toloss_function.initial_values
.- step_sizefloat
Step size for the gradient, labelled \(\alpha\) above. Default value is 1. Must be strictly positive.
- nestorov_weightfloat
The size of the look-ahead term in each iteration, labelled \(\beta\) above. Must be in the range
[0, 1]
, including the endpoints. The default value is0
, which implies that the momentum term is not active.- maxiterint
Maximum number of iterations. Default value is
5
.- ftolfloat
The tolerance for relative change in the loss function before termination. A termination can only be induced after at least 5 iterations. If
None
, termination only occurs oncemaxiter
iterations have been performed. Default value isNone
.- display_lossbool
If True, displays the change in loss at every iteration. Default is False.
- enforce_non_negativitybool
Enforces strict positivity on all the coefficients. Should only be used with local or scalar representations. Default value is
False
.
- property no_tqdm¶
Whether to avoid making a
tqdm
progress bar.
- optimize()[source]¶
Executes the optimization using the options stored in this class instance. The optimization will continue until convergence, or until the maximum number of iterations (
maxiter
) is exceeded.- Return type
- Returns
A
dict
of optimization results. See scipy.optimize.OptimizeResult for details. The entry'x'
, which contains the result, will be reshaped using the shape of the gradient fromloss_function
.