scipy.optimize.minimize: Methods, Bounds and Constraints

scipy.optimize.minimize finds the input that makes a function as small as possible:

from scipy.optimize import minimize

result = minimize(f, x0=[0, 0])    # f is your function, x0 is a starting guess

result.x          # the values that minimise it
result.success    # check this before trusting result.x

The starting guess matters, and so does result.success. A failed run still hands back an x, and it means nothing.

All output below is from real runs on Python 3.12.5, SciPy 1.18.1, NumPy 2.5.3.

Using scipy.optimize.minimize

Pass a function and a starting point. Everything else has a default:

from scipy.optimize import minimize

# a bowl with its lowest point at (3, -1)
def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

result = minimize(f, x0=[0, 0])

print("x       :", result.x.round(6))
print("fun     :", f"{result.fun:.2e}")
print("success :", result.success)
print("message :", result.message)
print("nit     :", result.nit, " iterations")
print("nfev    :", result.nfev, " function evaluations")

Output:

x       : [ 3. -1.]
fun     : 7.04e-15
success : True
message : Optimization terminated successfully.
nit     : 2  iterations
nfev    : 9  function evaluations

Your function must take a single array-like argument and return one number. If it needs extra arguments, they’re passed through args=.

x0 is a guess, not a constraint. A better guess means fewer iterations, and for a tricky function it can decide which minimum you land in.

What the OptimizeResult contains

minimize returns a bundle of information, not just the answer:

from scipy.optimize import minimize

def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

result = minimize(f, [0, 0])

print("everything the result carries:")
for key in result.keys():
    print("  ", key)

print()
print("the three you always check:")
print("  result.success :", result.success)
print("  result.x       :", result.x.round(6))
print("  result.fun     :", f"{result.fun:.3e}")

Output:

everything the result carries:
   fun
   jac
   hess_inv
   nfev
   njev
   status
   success
   message
   x
   nit

the three you always check:
  result.success : True
  result.x       : [ 3. -1.]
  result.fun     : 7.037e-15

Field	Meaning
x	The values that minimise the function
fun	The function value at that point
success	Whether the solver converged
message	Why it stopped, in words
nit	Iterations taken
nfev	How many times your function was called

nfev is the one to watch when your function is expensive. It counts every single call, including the ones used to estimate gradients numerically.

scipy minimize methods: which one to use

The method argument picks the algorithm, and they differ enormously in how much work they do:

import numpy as np
from scipy.optimize import minimize

def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

print(f"{'method':<14} {'x':<20} {'nfev':>5} {'success':>8}")
for name in ("Nelder-Mead", "Powell", "CG", "BFGS", "L-BFGS-B", "SLSQP", "TNC"):
    r = minimize(f, [0, 0], method=name)
    print(f"{name:<14} {str(r.x.round(4)):<20} {r.nfev:>5} {r.success!s:>8}")

print()
print("same answer, very different amounts of work")

Output:

method         x                     nfev  success
Nelder-Mead    [ 3. -1.]              142     True
Powell         [ 3. -1.]               33     True
CG             [ 3. -1.]                9     True
BFGS           [ 3. -1.]                9     True
L-BFGS-B       [ 3. -1.]                9     True
SLSQP          [ 3. -1.]                7     True
TNC            [ 3. -1.]                9     True

same answer, very different amounts of work

Method	Use when	Handles
BFGS	Smooth, unconstrained	The default without bounds
L-BFGS-B	Smooth with bounds	Bounds, large problems
SLSQP	Constrained problems	Bounds and constraints
Nelder-Mead	No usable gradient	Noisy or non-smooth functions
Powell	No gradient, more robust	Non-smooth functions
trust-constr	Hard constrained problems	Bounds, constraints, Hessians

You usually don’t pass method at all. SciPy picks BFGS, L-BFGS-B or SLSQP depending on whether you gave it bounds or constraints.

Reach for Nelder-Mead when your function is noisy or has kinks. It needs no gradient, which is exactly why it takes so many more evaluations.

Speeding minimize up with jac

Without a gradient, SciPy estimates one numerically, and that costs extra calls to your function:

from scipy.optimize import minimize, rosen, rosen_der

# the Rosenbrock function: a narrow curved valley, famously hard
without = minimize(rosen, [0, 0])
with_grad = minimize(rosen, [0, 0], jac=rosen_der)

print(f"{'':<12} {'iterations':>11} {'function calls':>15}")
print(f"{'no jac':<12} {without.nit:>11} {without.nfev:>15}")
print(f"{'with jac':<12} {with_grad.nit:>11} {with_grad.nfev:>15}")

print()
print("both land at", with_grad.x.round(6))
print("supplying the gradient removed the numerical estimation of it")

Output:

              iterations  function calls
no jac                19              72
with jac              19              24

both land at [0.999999 0.999998]
supplying the gradient removed the numerical estimation of it

The iteration count is identical. What changed is the 48 extra calls that were being used to approximate the gradient by finite differences.

If your function is slow, an analytic gradient is the single biggest win available. If deriving one is impractical, jax or autograd can produce it for you.

Adding bounds to scipy minimize

bounds restricts each variable to a range, given as a list of (low, high) pairs:

import numpy as np
from scipy.optimize import minimize

# the true minimum is at (3, -1), outside these bounds
def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

free = minimize(f, [0, 0])
limited = minimize(f, [0, 0], bounds=[(0, 2), (0, 2)])

print("unbounded :", free.x.round(4))
print("bounded   :", limited.x.round(4), " pressed against the limits")
print("success   :", limited.success)

print()
print("use None for a one-sided bound:")
one_sided = minimize(f, [0, 0], bounds=[(0, None), (None, 0)])
print("  ", one_sided.x.round(4))

Output:

unbounded : [ 3. -1.]
bounded   : [2. 0.]  pressed against the limits
success   : True

use None for a one-sided bound:
   [ 3. -1.]

The true minimum here is at (3, -1), outside the box. The bounded run stops at (2, 0), pressed against the nearest corner.

Use None for a side you don’t want to limit. (0, None) means “at least zero”, which is the usual way to keep a quantity positive.

Passing bounds changes the default method to L-BFGS-B, since BFGS cannot honour them. For linear problems there is linprog instead.

Constraints in scipy.optimize.minimize

Constraints are dictionaries, and the sign convention catches people out:

import numpy as np
from scipy.optimize import minimize

def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

# an equality constraint: the two values must add up to 1
equality = {"type": "eq", "fun": lambda x: x[0] + x[1] - 1}
r = minimize(f, [0, 0], constraints=equality)

print("constrained x:", r.x.round(6))
print("x0 + x1      :", round(r.x.sum(), 6), " exactly 1")

print()
# an inequality constraint: fun(x) >= 0 is what SciPy expects
inequality = {"type": "ineq", "fun": lambda x: 2 - x[0]}      # x0 <= 2
r2 = minimize(f, [0, 0], constraints=inequality)
print("x0 <= 2      :", r2.x.round(6))

Output:

constrained x: [ 2.5 -1.5]
x0 + x1      : 1.0  exactly 1

x0 <= 2      : [ 2. -1.]

• {"type": "eq", "fun": g} requires g(x) == 0.
• {"type": "ineq", "fun": g} requires g(x) >= 0, not <= 0.
• To express x0 <= 2, write the constraint as 2 - x[0].
• Pass a list of dictionaries when you have more than one.

Writing an inequality the wrong way round is the classic mistake. SciPy will happily optimise into the region you meant to exclude.

Why is scipy minimize result.success False?

A failed optimisation does not raise. It returns quietly with a meaningless answer:

import warnings
from scipy.optimize import minimize

# a straight line has no minimum, so the solver cannot converge
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    result = minimize(lambda x: x[0], x0=[0])

print("success :", result.success)
print("status  :", result.status)
print("message :", result.message)

print()
print("ALWAYS check result.success before using result.x")
print("a failed run still returns an x, and it is meaningless")
print("x was:", result.x.round(4))

Output:

success : False
status  : 2
message : Desired error not necessarily achieved due to precision loss.

ALWAYS check result.success before using result.x
a failed run still returns an x, and it is meaningless
x was: [-3.39211811e+155]

A straight line has no minimum, so the solver runs until it gives up. result.message explains why, and it’s worth reading rather than ignoring.

Check result.success every time. If it is False, try a different starting point, a different method, or rescale your variables so they are all of similar magnitude.

Watching minimize converge

The callback argument fires after every iteration, which makes the path visible:

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize

def f(x):
    return (x[0] - 3) ** 2 + (x[1] + 1) ** 2

# record every step the optimiser takes
path = [[0, 0]]
minimize(f, [0, 0], method="BFGS", callback=lambda xk: path.append(list(xk)))
path = np.array(path)

gx, gy = np.meshgrid(np.linspace(-1, 4.5, 200), np.linspace(-3, 2, 200))
gz = (gx - 3) ** 2 + (gy + 1) ** 2

plt.figure(figsize=(7.5, 5))
plt.contour(gx, gy, gz, levels=20, cmap="viridis", alpha=.7)
plt.plot(path[:, 0], path[:, 1], "o-", color="#c0392b", lw=2, label="optimiser path")
plt.plot(3, -1, "*", color="#0a7d32", markersize=18, label="true minimum")
plt.title("scipy.optimize.minimize walking downhill")
plt.xlabel("x0"); plt.ylabel("x1"); plt.legend(); plt.tight_layout()
plt.show()

print("steps taken:", len(path) - 1)

Output:

steps taken: 2