Differential Calculus and Optimization Basics › Multivariable Calculus

The portfolio and option models used here depend on multiple inputs. A portfolio's return depends on the returns of its constituent assets, with their weights specified. An option's model value depends on price, time, volatility, and interest rates simultaneously. This requires extending calculus to functions of multiple variables.

In single-variable calculus, the input has one independent coordinate, which can increase or decrease. With two or more input dimensions, there are infinitely many unit directions we could move. Understanding how the function changes in each of these directions requires multivariable calculus.

When a function depends on multiple variables, the partial derivative measures the rate of change with respect to one variable while holding all others constant.

The key conceptual insight is that partial derivatives treat other variables as if they were constants. If you want to know how portfolio return changes when you adjust the weight in one particular asset, you imagine all other weights frozen in place and observe how the return responds to that single change. This "one variable at a time" approach simplifies the analysis but also limits what we can learn. The true behavior of the function involves simultaneous changes in multiple variables, which we will address with gradients and directional derivatives.

The notation $\frac{\partial f}{\partial x_i}$ uses the curly "partial" symbol ∂ rather than the regular "d" used in single-variable calculus. This notational distinction is a reminder that we are holding other variables fixed. The computation itself follows the same rules as ordinary differentiation: we simply treat the other variables as constants and differentiate with respect to the variable of interest.

Consider a simple two-asset portfolio where total return $R$ depends on the returns $r_1$ and $r_2$ of each asset and their weights $w_1$ and $w_2$:

$$R(w_1, w_2, r_1, r_2) = w_1 r_1 + w_2 r_2$$

The partial derivatives tell us how sensitive total return is to each input:

$$\frac{\partial R}{\partial w_1} = r_1, \quad \frac{\partial R}{\partial r_1} = w_1$$

The first equation says that the sensitivity of portfolio return to the first weight equals the return of that asset. The second says that sensitivity to the first asset's return equals its weight in the portfolio. Both are intuitive, but partial derivatives make this reasoning precise.

These results have immediate practical interpretations. Holding $w_2$ fixed, increasing $w_1$ by 0.01 changes portfolio return by $0.01r_1$ but also increases net exposure and therefore requires additional financing. In a fully invested two-asset portfolio, $w_2=1-w_1$, so the relevant reallocation derivative is $dR/dw_1=r_1-r_2$: shifting 0.01 from asset 2 to asset 1 changes return by $0.01(r_1-r_2)$. Separately, if asset 1's return changes by 0.01 while its weight is 0.40, portfolio return changes by 0.004, or 0.4 percentage points. The same local linearization idea also appears in factor-based risk decompositions.

For a linear factor-return model $R=\beta^\mathsf{T}F$, $F$ is the vector of factor returns and $\beta$ is the vector of held-fixed exposures. A factor move $\Delta F$ changes return by $\beta^\mathsf{T}\Delta F$ exactly. For a differentiable nonlinear factor model, the corresponding local gradient gives a first-order approximation instead.

Collecting all partial derivatives of a function into a single vector gives us the gradient, a fundamental object in optimization.

The gradient represents the multivariable generalization of the derivative. For a differentiable function, it describes first-order changes from simultaneous input moves. A nonzero gradient points toward local ascent in the chosen Euclidean coordinate scaling; its negative points toward local descent. Whether either move improves the objective depends on whether we are maximizing or minimizing and whether the direction is feasible.

Assume $f$ is differentiable at the point, so a small input vector $d$ satisfies $f(x+d)=f(x)+\nabla f(x)^\mathsf{T}d+o(\|d\|)$: the remainder becomes small relative to the size of the input move. Among Euclidean unit directions, $\nabla f/\|\nabla f\|$ maximizes the directional derivative when the gradient is nonzero. Its magnitude is that maximum first-order rate of increase. Both the direction and norm depend on coordinate scaling, so inputs with different units need a specified scaling before those comparisons are meaningful.

To understand this result, consider the directional derivative $D_u f=\nabla f^\mathsf{T}u$ for a Euclidean unit vector $u$. The Cauchy-Schwarz inequality gives $D_u f\leq\|\nabla f\|$, with equality in the normalized gradient direction when the gradient is nonzero; the opposite direction gives steepest descent. If the gradient is zero, every first-order directional derivative is zero, so no unique steepest direction exists.

The magnitude $\|\nabla f\| = \sqrt{\sum_i (\partial f / \partial x_i)^2}$ measures the maximum first-order rate at that point under this differentiability and scaling convention. A small value does not imply small curvature or a uniformly flat surrounding region. A zero gradient identifies a stationary point with no preferred first-order direction; it may be a minimum, maximum, saddle, or degenerate stationary point requiring further analysis.

While analytical derivatives are preferred when available, we often need to compute derivatives numerically, especially for complex models or when working with black-box functions.

Sometimes we have access to a function only through evaluations: we can supply inputs and observe outputs, but cannot differentiate an accessible expression. This can arise with simulation-based models, evaluation-only software interfaces, or nested pricing calculations. Numerical differentiation approximates derivatives using function evaluations. Nested calculations alone do not rule out automatic differentiation when the underlying code is accessible.

The simplest approach uses the finite difference approximation:

$$f'(x) \approx \frac{f(x + h) - f(x)}{h}$$

For sufficiently smooth functions, the central difference formula has higher truncation order:

$$f'(x) \approx \frac{f(x + h) - f(x - h)}{2h}$$

where:

• $f'(x)$: the derivative we are approximating
• $h$: a small step size (often around $10^{-6}$ to $10^{-5}$ for central differences in double precision, depending on scaling)
• $f(x + h)$ and $f(x - h)$: function evaluations at points symmetric around $x$

For smooth functions with the required derivatives, the central difference has $O(h^2)$ truncation error compared to $O(h)$ for the forward difference. This compares truncation orders, not total numerical errors: choosing $h$ involves a trade-off between truncation error, evaluation error, and floating-point cancellation.

The Taylor expansions show where the higher truncation order comes from. In the expansions of $f(x+h)$ and $f(x-h)$, the first-order terms have opposite signs, so subtraction produces $2h f'(x)$. The even-order terms have the same signs and cancel. In particular,

$$\frac{f(x+h)-f(x-h)}{2h}=f'(x)+\frac{h^2}{6}f'''(x)+O(h^4),$$

when the required derivatives exist near $x$. Thus the central quotient has $O(h^2)$ truncation error, whereas the forward difference generally has $O(h)$ truncation error.

The choice of step size $h$ balances truncation error against evaluation error and cancellation. The indicative $10^{-6}$ to $10^{-5}$ range assumes well-scaled, smooth functions and accurate deterministic double-precision evaluations; different coordinates can need different steps. Simulation noise is amplified by division by $h$, and both evaluation points must remain in the function's domain. A finite quotient alone does not establish that a derivative exists.