Python Scipy Minimize [With 8 Examples]

In this Python tutorial, we will learn about the “Python Scipy Minimize“, where we will know how to find the minimum value of a given function and cover the following topics.

• Python Scipy Minimize
• Python Scipy Minimize Multiple Variables
• Python Scipy Minimize Bounds
• Python Scipy Minimize Constraints
• Python Scipy Minimize Scalar
• Python Scipy Minimize Powell
• Python Scipy Minimize Not Working
• Python Scipy Minimize Trust-Constr

Python Scipy Minimize

The Python Scipy module scipy.optimize has a method minimize() that takes a scalar function of one or more variables being minimized.

The syntax is given below.

scipy.optimize.minimize(fun, x0, method=None, args=(),  jac=None, hessp=None, hess=None, constraints=(), tol=None, bounds=None, callback=None, options=None)

Where parameters are:

• fun(callable): To minimize is the objective function.
• x0(shape(n), ndarray): First intuition. an array of real objects, where n is the total number of independent variables, and the size of the array is (n,).
• method(str or callable): Solver type ought to be one of trust-Krylov, Nelder-Mead, CG, Powell, BFGS, L-BFGS-B, TNC, COBYLA,trust-exact, Newton-CG, SLSQP, dogleg, trust-ncg, trust-constr.
• args(tuple): Additional arguments supplied to the derivatives of the objective function.
• jac(bool, cs , 2 or 3 point): An approach to calculating the gradient vector. only for BFGS, CG, L-BFGS-B, Newton-CG, TNC, dogleg, SLSQP, trust-ncg, trust-exact,trust-krylov, and trust-constr If it’s a callable, it ought to be a function that gives back the gradient vector: “jac(x, *args) -> array_like, shape (n,)”. Where x is an array with the shape (n,), and args is a tuple of the fixed parameters. The objective function and gradient are considered to be returned by fun if jac is a Boolean and is True.
• hessp(callable): Hessian of the objective function multiplied by p, a random vector. Specifically for Newton-CG, trust-ncg, trust-krylov, and trust-constr. Hessp or hess must only be given once. Hessp will be disregarded if hess is supplied. The Hessian must be multiplied by any vector using hessp: “hessp(x, p, *args) -> ndarray shape (n,)”. Where args is a tuple containing the fixed parameters, p is an arbitrary vector of dimension (n), and x is an (n,) ndarray.
• hess: The Hessian matrix computation method. only for the Dogleg, Newton-CG, Trust-NCG, Trust-Exact, Trust-Krylov, and Trust-Constr algorithms. This should output the Hessian matrix if it is callable: “ess(x, *args) -> {LinearOperator, spmatrix, array}, (n, n)”. Where args is a tuple of the fixed parameters and x is an array of size (n, n). To choose a finite difference scheme for the numerical estimation of the hessian, the keywords “2-point,” “3-point,” and “cs” can also be used.
• constraints(dict,constraint): limits the definition. only in relation to SLSQP, COBYLA, and trust-constr. A single object or a set of objects that specify constraints for the optimization problem are referred to as “trust-constr” constraints. Available constraints are: NonLinear or Linear Constraints.
• tol(float): Tolerance for ending. When tol is supplied, the chosen minimization algorithm equalizes all pertinent solver-specific tolerances with tol. Use solver-specific settings for fine-grained control.
• bounds(bounds or sequence): For the L-BFGS-B, Nelder-Mead, TNC, Powell, SLSQP, and trust-constr techniques, bounds on the variables. The boundaries can be specified in one of two ways: Instance of the class Bounds and for each element in x, a list of (min, max) pairs is given. To specify no bound, use the word none.
• callback(): After each iteration, called. It is callable with the signature for “trust-constr.”
• options(dict): A list of possible solvers.

The method minimize() returns res(A OptimizeResult object is used to represent the optimization result. The solution array x, success, a Boolean indication indicating if the optimizer successfully terminated, and message, which explains the termination reason, are important features).

Let’s take an example by following the below step:

Let’s think about the Rosenbrock function minimization issue. Rosen uses this function and its corresponding derivatives.

Import the required method or libraries using the below python code.

from scipy import optimize

An easy way to use the Nelder-Mead approach is using the below code.

data_x0 = [2.1, 0.5, 0.9, 1.7, 1.1]
result = optimize.minimize(optimize.rosen, data_x0, method='Nelder-Mead', tol=1e-5)
result.x

This is how to use the method minimize() Python Scipy to minimize the function with different methods.

Read: Python Scipy Chi-Square Test

Python Scipy Minimize Multiple Variables

Here in this section, we will create a method manually that will take several parameters or variables, to find the minimum value of the function using the method minimize() of module scipy.optimize. Follow the below steps to create a method.

Import the required libraries using the below python code.

from scipy import optimize

Define the method using the below code.

def fun(paramt):
    # print(paramt)  # <-- As you can see, params is an array in NumPy.
    x, y, z = paramt # <-- You might want to give the component variables names to make them easier to read.
    return x**2 + y**3 + z**3

Define the initial guess and pass the guess or function to a method minimize() using the below code.

first_guess = [0.5, 0.5, 0.5]
res = optimize.minimize(fun, first_guess)

Check the result or values for the several variables that we defined in the function using the below code.

res.x

This is how to find the minimum value for multiple variables by creating a method in Python Scipy.

Read: Python Scipy Matrix + Examples

Python Scipy Minimize Bounds

The Python Scipy module scipy.optimize contains a method Bounds() that defined the bounds constraints on variables.

The constraints takes the form of a general inequality : lb <= x <= ub

The syntax is given below.

scipy.optimize.Bounds(lb, ub, keep_feasible=False)

Where parameters are:

• ub, lb(array_data): Independent variable lower and upper boundaries. Each array needs to be the same size as x or it must be a scalar, in which case a bound will apply to all the variables equally. To fix a variable, equalize the parts of lb and ub. To remove boundaries from all or some variables, use np.inf along with the appropriate sign. Remember that you can mix constraints of different types, such as interval, one-sided, or equality, by adjusting the various lb and ub components as required.
• keep_feasible(boolean): Whether to continue making the constraint elements workable after iterations. This property was set for all components with a single value. False is the default. has no impact on equality restrictions.

Let’s take an example by following the below steps:

Import the required method and define the bound using the below python code.

from scipy.optimize import Bounds

Define the bounds using the below code.

Bounds(2,7)

This is how to define the bounds using the method Bounds() of Python Scipy.

Read: Scipy Linalg – Helpful Guide

Python Scipy Minimize Constraints

Here in this section, we will create constraints and pass the constraints to a method scipy.optimize.minimize() of Python Scipy.

Define the constraints using the below python code.

s[0] + s[1] = 1

Creating a function that must equal zero would be an equality (type=’eq’) constraint using the below code.

def cont(s):
    return s[0] + s[1] - 1

Then, we create a dict of your constraint (or, if there are multiple, a list of dicts) using the below code.

constarnt = {'type':'eq', 'fun': cont}
const_real(t):
    return np.sum(np.iscomplex(s))

And be sure to mention both constraints using the below code.

constarnt = [{'type':'eq', 'fun': const},
        {'type':'eq', 'fun': const_real}]

Next, we input constraints into minimizing method as shown in the below code.

scipy.optimize.minimize(function, data_x0, constraints=constarnt)

This is how to input the constraints into the method minimize().

Read: Scipy Stats Zscore + Examples

Python Scipy Minimize Scalar

The Python Scipy module scipy.optimize.minimize contains a method minimize_scalar() that takes the scalar function of one variable that needs to minimize. If your function is a one-variable scalar function, you can use the minimize_scalar() function to get the function’s minimum value and the value that minimizes it.

The syntax is given below.

scipy.optimize.minimize_scalar(fun, bounds=None, args=(), bracket=None, method='brent', options=None, tol=None)

Where parameters are:

• fun(callable): Objective function. A scalar must be returned by a scalar function.
• bounds(sequence): Bounds are a required parameter for the method “bounded” and have to contain the two items that make up the optimization bounds.
• args(tuple): Additional inputs given to the objective function.
• bracket(sequence): For the methods “golden” and “brent,” the bracketing interval is defined by the bracket, which may have three items (a, b, and c) so that a b c, and fun(b) fun(a), fun(c), or two items (a and c), which are taken to be the starting interval for a downhill bracket search. It’s not always a given that the solution will meet the condition a == x = c.
• method: Solver type. ought to be one of Golden, Bounded, and Brent.
• options(dict): A list of possible solvers.
• tol(float): Tolerance for terminating. Use solver-specific settings for fine-grained control.

The method minimize() returns res(A OptimizeResult object is used to represent the optimization result).

Let’s understand with an example by following the below steps:

Import the required method or libraries using the below python code.

from scipy import optimize

Create a function that we are going to minimize using the below code.

def fun(s):
    return (s - 3) * s * (s + 3)**3

Pass the above function to a method minimize_scalar() to find the minimum value using the below code.

result = optimize.minimize_scalar(fun)
result.x

This is how to apply the method minimize_scalar() on the function to find the minimum value.

Read: Scipy Signal – Helpful Tutorial

Python Scipy Minimize Powell

The Python Scipy method minimize() that we have learned above sub-section accepts the method Powell that uses a modified version of Powell’s technique to minimize a scalar function of one or more variables.

The syntax is given below.

scipy.optimize.minimize(fun, x0,method='Powell', bounds=None, 'xtol': 0.0002, 'ftol': 0.0002, 'maxiter': None, 'maxfev': None, 'disp': False, 'direc': None, 'return_all': False})

Where parameters are:

• disp(boolean): To print convergence messages, set to True.
• xtol(float): Acceptable relative error in the xopt solution for convergence.
• ftol(float): Relative error is fun(xopt) acceptable for convergence.
• maxiter, maxfev(int): Maximum number of function evaluations and iterations permitted. If neither maxiter nor maxfev is set, the default value is N*1000, where N is the number of variables. The minimization process will end when the first value is reached if both maxiter and maxfev are set.
• direc(ndarray): Initial set of Powell technique direction vectors.
• return_all(boolean): If True is selected, the best solutions for each iteration will be returned in a list.

For the rest of the parameters, please visit the first section of this tutorial.

Read: Scipy Rotate Image + Examples

Python Scipy Minimize Not Working

If we find that method minimize() is not working, which means any provided input or parameters, etc, aren’t provided in the way that they should be. Sometimes we provide vectors in place of scalars to a method, or invalid parameters and functions.

This kind of mistake generates an error or tells that the minimize not working. To avoid the error, follow the proper documentation about the method minimize() how to use this method, and what kind of valid value or parameters it accepts. For demonstration purposes, there is an error on StackOverflow.

Python Scipy Minimize Trust-Constr

The method trust-exact is compatible with the Python Scipy function minimize (), which we learned about in the previous section. Using a trust-exact method with a function minimize() that is almost accurate to minimize the scalar function of one or more variables.

The syntax is given below.

scipy.optimize.minimize(fun, x0, args=(), method='trust-exact', jac=None, hess=None, tol=None, callback=None, options={})

Where parameters are:

• intial_tr_radius(float): Initial radius of trust-region.
• max_tr_radius(float): Radius of the trust-region at its maximum value. Over this value, no additional steps will be suggested.
• eta(float): Acceptance criteria for proposed steps that are dependent on trust regions.
• gtol(float): Before a termination is successful, the gradient norm must be lower than the gtol.

For the rest of the parameters, please refer to the first section of this tutorial.

You may also like to read the following Python SciPy tutorial.

• Scipy Optimize – Helpful Guide
• Python Scipy Stats Skew
• Python Scipy Stats Mode with Examples
• Python Scipy Stats Multivariate_Normal
• Scipy Constants – Multiple Examples
• Scipy Sparse – Helpful Tutorial
• Scipy Rotate Image + Examples

So, in this tutorial, we have learned about “Python Scipy Minimize” and covered the following topics.

• Python Scipy Minimize
• Python Scipy Minimize Multiple Variables
• Python Scipy Minimize Bounds
• Python Scipy Minimize Constraints
• Python Scipy Minimize Scalar
• Python Scipy Minimize Powell
• Python Scipy Minimize Not Working
• Python Scipy Minimize Trust-Constr