Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Differential Calculus and Optimization Basics › Practical Implementation with SciPy

Michael BrenndoerferOctober 24, 20255 min read

Quantitative FinanceAll chapters

Part 1 of 15Mathematical Foundations

Chapter 6Differential Calculus and Optimization Basics › Practical Implementation with SciPy

Reading section 7 of 11 · Practical Implementation with SciPy · 5 minSection 7 of 11 · 5 min

Section 7 of 11 · 5 min read

Differential Calculus and Optimization Basics › Practical Implementation with SciPyOne section · Sections in this chapter7 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

SciPy's optimize module implements methods for many smooth unconstrained and constrained examples. Suitability for a financial application depends on the objective, constraints, derivatives, scaling, conditioning, and the guarantees required; always inspect solver status and residuals.

In[33]:

Code

import numpy as np
from scipy.optimize import minimize


# Example 1: Unconstrained optimization
def rosenbrock(x):
    """The Rosenbrock function - a classic test function.

    Known for its curved valley that makes optimization challenging.
    Global minimum at x = [1, 1, ..., 1] with f(x*) = 0.
    """
    return sum(100.0 * (x[1:] - x[:-1] ** 2.0) ** 2.0 + (1 - x[:-1]) ** 2.0)


def rosenbrock_gradient(x):
    """Gradient for the two-dimensional Rosenbrock example."""
    return np.array(
        [
            -400 * x[0] * (x[1] - x[0] ** 2) - 2 * (1 - x[0]),
            200 * (x[1] - x[0] ** 2),
        ]
    )


# Different optimization methods
x0 = np.array([-1.0, -1.0])

results = {}
for method in ["Nelder-Mead", "BFGS", "CG"]:
    result = minimize(rosenbrock, x0, method=method)
    results[method] = {
        "x": result.x,
        "fun": result.fun,
        "nfev": result.nfev,
        "success": result.success,
        "message": str(result.message),
        "gradient_norm": np.linalg.norm(rosenbrock_gradient(result.x)),
    }

Out[34]:

Console

Optimization Results on Rosenbrock Function
==================================================
True minimum: [1, 1], f(x*) = 0

Nelder-Mead:
  Solution: [0.999999, 0.999995]
  Function value: 5.31e-10
  Function evaluations: 125
  Solver success: True (Optimization terminated successfully.)
  Gradient norm: 1.03e-03

BFGS:
  Solution: [0.999996, 0.999991]
  Function value: 2.00e-11
  Function evaluations: 120
  Solver success: True (Optimization terminated successfully.)
  Gradient norm: 5.92e-06

CG:
  Solution: [0.999997, 0.999995]
  Function value: 7.46e-12
  Function evaluations: 210
  Solver success: False (Desired error not necessarily achieved due to precision loss.)
  Gradient norm: 4.19e-05

BFGS is a quasi-Newton method that uses first derivatives and builds an inverse-Hessian approximation. Nelder-Mead is derivative-free. These calls do not supply jac, so BFGS and CG estimate derivatives numerically; the analytic gradient above is used to report the final residual. Compare evaluation counts together with stopping status and residuals, not as a universal ranking. A small objective value does not establish successful termination: if CG reports precision loss, its candidate still requires investigation. Relative performance depends on scaling, dimension, noise, initialization, tolerances, and the cost metric.

Handling Bounds and ConstraintsLink Copied

Real financial problems often involve bounds (no short selling: $w \geq 0$ ) and multiple constraints (budget, sector limits, etc.).

The next 9% target example has strictly interior weights. Its first-order check fits equality multipliers only, which is valid when all box bounds are inactive. Active-bound portfolios need the corresponding KKT bound multipliers or a feasible-direction test; the code explicitly declines to certify them with this interior-only check.

In[35]:

Code

from scipy.optimize import Bounds

# Three-asset portfolio optimization with constraints
n_assets = 3
mu_3 = np.array([0.12, 0.08, 0.05])
sigma_3 = np.array([0.22, 0.15, 0.08])
corr_matrix = np.array([[1.0, 0.4, 0.2], [0.4, 1.0, 0.3], [0.2, 0.3, 1.0]])
cov_3 = np.outer(sigma_3, sigma_3) * corr_matrix


def port_var_3(w):
    return w @ cov_3 @ w


def port_ret_3(w):
    return w @ mu_3


# Constraints
target_ret = 0.09
constraints = [
    {"type": "eq", "fun": lambda w: np.sum(w) - 1},  # Fully invested
    {
        "type": "eq",
        "fun": lambda w: port_ret_3(w) - target_ret,
    },  # Target return
]

# Bounds: no short selling (all weights >= 0)
bounds = Bounds(lb=np.zeros(n_assets), ub=np.ones(n_assets))

# Optimize
w0 = np.ones(n_assets) / n_assets
result = minimize(
    port_var_3,
    w0,
    method="SLSQP",
    bounds=bounds,
    constraints=constraints,
    options={"ftol": 1e-12, "maxiter": 500},
)
w_optimal = result.x

if (
    not result.success
    or not np.all(np.isfinite(w_optimal))
    or not np.isfinite(result.fun)
):
    raise RuntimeError(f"Portfolio optimization failed: {result.message}")
if np.any(w_optimal <= bounds.lb + 1e-8) or np.any(w_optimal >= bounds.ub - 1e-8):
    raise RuntimeError(
        "Interior-only stationarity check is not applicable; active bounds need KKT analysis"
    )

# Check solver status, feasibility, bounds, and first-order stationarity.
equality_residuals = np.array(
    [constraint["fun"](w_optimal) for constraint in constraints]
)
lower_bound_violation = np.maximum(bounds.lb - w_optimal, 0).max()
upper_bound_violation = np.maximum(w_optimal - bounds.ub, 0).max()
constraint_jacobian = np.vstack([np.ones(n_assets), mu_3])
objective_gradient = 2 * cov_3 @ w_optimal
lagrange_multipliers = np.linalg.lstsq(
    constraint_jacobian.T, -objective_gradient, rcond=None
)[0]
stationarity_residual = np.linalg.norm(
    objective_gradient + constraint_jacobian.T @ lagrange_multipliers
)

checks_passed = (
    result.success
    and np.max(np.abs(equality_residuals)) < 1e-8
    and max(lower_bound_violation, upper_bound_violation) < 1e-10
    and stationarity_residual < 1e-8
)
if not checks_passed:
    raise RuntimeError(f"Portfolio optimization checks failed: {result.message}")

Out[36]:

Console

Three-Asset Portfolio Optimization (No Short Sales)
==================================================
Solver success: True (Optimization terminated successfully)
Maximum equality residual: 4.94e-13
Maximum bound violation: 0.00e+00
First-order stationarity residual: 5.05e-11

Asset Parameters:
Asset    Return     Std Dev   
1          12.0%      22.0%
2           8.0%      15.0%
3           5.0%       8.0%

Optimal Weights for 9% Target Return:
  Asset 1:  43.86%
  Asset 2:  31.00%
  Asset 3:  25.14%

Portfolio Statistics:
  Expected Return: 9.00%
  Standard Deviation: 12.96%
  Sharpe Ratio (rf=2%): 0.540

The successful solver result allocates across all three assets. The reported equality, bound, and first-order residuals verify this interior solution before we describe it as the minimum-variance portfolio for the 9% target under the stated constraints. The displayed expected Sharpe ratio is $(\mu_p-r_f)/\sigma_p$ : modeled expected excess return divided by modeled return standard deviation, with portfolio inputs and the 2% risk-free return measured over the same horizon. We minimized variance at a fixed expected-return target, not the Sharpe ratio directly.