QP Portfolio › Formula

Michael BrenndoerferSeptember 7, 202517 min read
Machine Learning from ScratchAll chapters
Part 9 of 9Optimization
Chapter 64QP Portfolio › Formula
Reading section 2 of 8 · Formula · 17 minSection 2 of 8 · 17 min

Book contents

Section 2 of 8 · 17 min read

QP Portfolio › FormulaOne section · Sections in this chapter2 / 8 ↓
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Building Intuition: Why Portfolio Risk is More Than the Sum of Its Parts

Imagine you're investing in two assets: a technology stock and a utility stock. If you put 50% of your money in each, you might naively think your portfolio risk is simply the average of their individual risks. But this intuition misses something important: how these assets move together matters just as much as how risky each one is individually.

When technology stocks crash, utility stocks often remain stable (or even rise slightly) because they represent different economic sectors. This negative correlation means your portfolio is less risky than the simple average would suggest, as the losses in one asset are partially offset by stability in the other. Conversely, if both assets tend to move together (positive correlation), your portfolio risk is amplified beyond the simple average.

This fundamental insight, that portfolio risk depends on both individual asset risks and their relationships, is why we need a mathematical framework that captures these interactions. Linear models simply cannot represent this reality, which is why portfolio optimization requires quadratic programming.

Out[3]:
Visualization
Portfolio volatility curves showing diversification benefits with different asset correlations from -0.8 to +0.8.
The diversification effect for a two-asset portfolio where both assets have 20% individual volatility. With high positive correlation (ρ = +0.8), portfolio risk stays close to the naive average. With zero or negative correlation, combining assets reduces risk below the average, with the minimum-risk portfolio achieving lower volatility than either individual asset.

From Intuition to Mathematics: The Portfolio Variance Formula

Now that we've seen how correlation shapes portfolio risk, let's build the mathematical framework that captures this insight. We'll proceed step by step, starting with the most intuitive representation and gradually moving toward the matrix formulation that powers modern optimization algorithms.

Step 1: Understanding What We're Measuring

Portfolio variance measures how much the portfolio's return fluctuates over time. A high variance means the portfolio value experiences large swings, while low variance indicates more stable returns. Our goal is to construct a portfolio that achieves our desired return while minimizing this variance.

Step 2: The Building Blocks

Before we can express portfolio variance mathematically, we need to define our components:

  • wiw_i: The weight (proportion) of asset ii in the portfolio, where i∈{1,2,…,n}i \in \{1, 2, \ldots, n\}. If wi=0.3w_i = 0.3, then 30% of our capital is invested in asset ii. These weights must satisfy ∑i=1nwi=1\sum_{i=1}^{n} w_i = 1 (100% of capital allocated).
  • σij\sigma_{ij}: The covariance between assets ii and jj, where i,j∈{1,2,…,n}i, j \in \{1, 2, \ldots, n\}. This measures how the returns of these two assets move together. When i=ji = j, we have σii=σi2\sigma_{ii} = \sigma_i^2, which is the variance of asset ii (how much asset ii's return fluctuates on its own). Note that σij=σji\sigma_{ij} = \sigma_{ji} (covariance is symmetric).
  • μi\mu_i: The expected return of asset ii (the average return we anticipate), where i∈{1,2,…,n}i \in \{1, 2, \ldots, n\}.
  • nn: The total number of assets in the portfolio.

Step 3: Why a Double Summation?

The portfolio variance formula uses a double summation to capture every possible interaction between assets:

σp2=∑i=1n∑j=1nwiwjσij\sigma_p^2 = \sum_{i=1}^{n} \sum_{j=1}^{n} w_i w_j \sigma_{ij}

Let's break this down to understand why both summations are necessary:

  1. When i=ji = j: We're looking at terms like wiwiσii=wi2σi2w_i w_i \sigma_{ii} = w_i^2 \sigma_i^2, where σii=σi2\sigma_{ii} = \sigma_i^2 is the variance of asset ii. This captures how each asset's own variance contributes to portfolio risk, weighted by the square of its portfolio weight. Notice the squared weight. This is important because doubling your position in an asset quadruples its contribution to portfolio variance (risk grows quadratically with position size).

  2. When i≠ji \neq j: We're looking at terms like wiwjσijw_i w_j \sigma_{ij} where i≠ji \neq j. These cross-terms capture the diversification effects. If σij<0\sigma_{ij} < 0 (negative correlation), this term reduces portfolio variance. If σij>0\sigma_{ij} > 0 (positive correlation), it increases variance. If σij=0\sigma_{ij} = 0 (uncorrelated assets), the term contributes zero to portfolio variance.

The double summation includes all n2n^2 possible asset pairs in addition to the nn individual assets. Accounting for both types of terms makes the formula mathematically complete and useful in practice.

Out[4]:
Visualization
Covariance matrix heatmap showing diagonal variance terms in blue and off-diagonal covariance terms in green, illustrating portfolio variance calculation.
Visual representation of the double summation in portfolio variance. The 5 blue diagonal cells represent variance contributions from each asset individually, while the 20 green off-diagonal cells represent covariance terms that capture diversification effects. When covariances are negative, these off-diagonal terms reduce portfolio variance.

Step 4: The Expected Return Formula

While portfolio variance requires the complexity of a double summation, expected portfolio return is simple. It's just the weighted average:

μp=∑i=1nwiμi\mu_p = \sum_{i=1}^{n} w_i \mu_i

where:

  • μp\mu_p: The expected return of the portfolio (a scalar)
  • wiw_i: The weight of asset ii in the portfolio
  • μi\mu_i: The expected return of asset ii

This linear relationship follows directly from the definition of expected value. Consider a two-asset example:

  • Asset 1: weight w1=0.3w_1 = 0.3 (30%), expected return μ1=0.10\mu_1 = 0.10 (10%)
  • Asset 2: weight w2=0.7w_2 = 0.7 (70%), expected return μ2=0.05\mu_2 = 0.05 (5%)

The portfolio's expected return is:

μp=w1μ1+w2μ2=(0.3)(0.10)+(0.7)(0.05)=0.03+0.035=0.065\mu_p = w_1 \mu_1 + w_2 \mu_2 = (0.3)(0.10) + (0.7)(0.05) = 0.03 + 0.035 = 0.065

which equals 6.5%. Unlike risk, expected returns combine linearly with no cross-terms or interaction effects.

The Matrix Formulation

The double summation notation reveals the structure of portfolio variance, but writing out n2n^2 terms becomes unwieldy as the number of assets grows. Matrix notation provides the compact representation that optimization algorithms require while preserving the same mathematical content.

Defining Our Vectors and Matrices

We organize our problem into three mathematical objects:

  • ww: The n×1n \times 1 column vector of portfolio weights: w=[w1,w2,…,wn]Tw = [w_1, w_2, \ldots, w_n]^T, where wiw_i is the weight of asset ii and ∑i=1nwi=1\sum_{i=1}^{n} w_i = 1.
  • μ\mu: The n×1n \times 1 column vector of expected returns: μ=[μ1,μ2,…,μn]T\mu = [\mu_1, \mu_2, \ldots, \mu_n]^T, where μi\mu_i is the expected return of asset ii.
  • Σ\Sigma: The n×nn \times n covariance matrix where entry Σij=σij\Sigma_{ij} = \sigma_{ij} for i,j∈{1,2,…,n}i, j \in \{1, 2, \ldots, n\}.

The covariance matrix Σ\Sigma is symmetric (since σij=σji\sigma_{ij} = \sigma_{ji}, which implies Σij=Σji\Sigma_{ij} = \Sigma_{ji}) and positive semi-definite (required for valid covariance matrices). The diagonal elements Σii=σii=σi2\Sigma_{ii} = \sigma_{ii} = \sigma_i^2 are the individual asset variances, while off-diagonal elements Σij=σij\Sigma_{ij} = \sigma_{ij} (for i≠ji \neq j) capture the relationships between different assets.

The Matrix Form of Portfolio Variance

In matrix notation, the portfolio variance becomes compact:

σp2=wTΣw\sigma_p^2 = w^T \Sigma w

This single matrix multiplication is mathematically equivalent to the double summation, but it's far more efficient to compute. To see the equivalence, we expand the matrix multiplication step by step:

  1. Compute Σw\Sigma w (an n×1n \times 1 vector): The ii-th element of this vector is:
(Σw)i=∑j=1nΣijwj=∑j=1nσijwj(\Sigma w)_i = \sum_{j=1}^{n} \Sigma_{ij} w_j = \sum_{j=1}^{n} \sigma_{ij} w_j
  1. Compute wT(Σw)w^T (\Sigma w) (a scalar): This is the dot product of ww with Σw\Sigma w:
wT(Σw)=∑i=1nwi(Σw)i=∑i=1nwi(∑j=1nσijwj)w^T (\Sigma w) = \sum_{i=1}^{n} w_i (\Sigma w)_i = \sum_{i=1}^{n} w_i \left(\sum_{j=1}^{n} \sigma_{ij} w_j\right)
  1. Distribute the outer sum: Since scalar multiplication distributes over addition:
wTΣw=∑i=1n∑j=1nwiσijwj=∑i=1n∑j=1nwiwjσijw^T \Sigma w = \sum_{i=1}^{n} \sum_{j=1}^{n} w_i \sigma_{ij} w_j = \sum_{i=1}^{n} \sum_{j=1}^{n} w_i w_j \sigma_{ij}

This confirms that the matrix form wTΣww^T \Sigma w equals the summation form ∑i=1n∑j=1nwiwjσij\sum_{i=1}^{n} \sum_{j=1}^{n} w_i w_j \sigma_{ij}. The matrix notation makes the quadratic nature explicit: we're computing a quadratic form, which is why this is a quadratic programming problem rather than a linear one.

The Matrix Form of Expected Return

Similarly, the expected return becomes a simple dot product:

μp=wTμ=∑i=1nwiμi\mu_p = w^T \mu = \sum_{i=1}^{n} w_i \mu_i

The Complete Quadratic Programming Formulation

With both variance (wTΣww^T \Sigma w) and expected return (wTμw^T \mu) expressed in matrix form, we can now state the complete optimization problem. The goal is to find portfolio weights that minimize risk while achieving our investment objectives:

min⁡w12wTΣw\min_{w} \quad \frac{1}{2} w^T \Sigma w

subject to:

∑i=1nwi=1(budget constraint)\sum_{i=1}^{n} w_i = 1 \quad \text{(budget constraint)} ∑i=1nwiμi=μtarget(return constraint)\sum_{i=1}^{n} w_i \mu_i = \mu_{target} \quad \text{(return constraint)} wi≥0∀i(no short selling)w_i \geq 0 \quad \forall i \quad \text{(no short selling)}

Understanding the Objective Function

The factor of 12\frac{1}{2} in the objective function is a mathematical convenience that simplifies derivatives without changing the optimal solution. To see why, consider the gradient (vector of partial derivatives):

∇w(12wTΣw)=12⋅2Σw=Σw\nabla_w \left(\frac{1}{2} w^T \Sigma w\right) = \frac{1}{2} \cdot 2 \Sigma w = \Sigma w

where:

  • ∇w\nabla_w: The gradient operator with respect to ww (produces an n×1n \times 1 vector of partial derivatives)
  • Σw\Sigma w: The result of multiplying the covariance matrix by the weight vector

We use the fact that ∇w(wTΣw)=2Σw\nabla_w(w^T \Sigma w) = 2\Sigma w for symmetric Σ\Sigma. The factor of 2 from the gradient cancels with the 12\frac{1}{2} in the objective, giving us the clean result Σw\Sigma w. Without the 12\frac{1}{2} factor, we would have ∇w(wTΣw)=2Σw\nabla_w(w^T \Sigma w) = 2\Sigma w, which is less convenient. Since we're minimizing, multiplying the objective by a positive constant (12\frac{1}{2}) doesn't change the optimal solution. It only scales the objective value.

Understanding the Constraints

Each constraint serves a specific practical purpose:

  • Budget constraint (∑i=1nwi=1\sum_{i=1}^{n} w_i = 1): Ensures we invest exactly 100% of our available capital, no more and no less. This prevents the optimizer from suggesting we hold cash or borrow money (unless we explicitly allow it). The constraint can be written in matrix form as 1Tw=1\mathbf{1}^T w = 1, where 1\mathbf{1} is a vector of ones.

  • Return constraint (∑i=1nwiμi=μtarget\sum_{i=1}^{n} w_i \mu_i = \mu_{target}): Ensures our portfolio achieves a specific expected return target μtarget\mu_{target}. In matrix form, this is wTμ=μtargetw^T \mu = \mu_{target}. By varying μtarget\mu_{target}, we can trace out the efficient frontier, which is the set of portfolios that offer the best risk-return trade-off.

  • Non-negativity constraints (wi≥0w_i \geq 0 for all i∈{1,2,…,n}i \in \{1, 2, \ldots, n\}): Prevents short selling (negative weights). In practice, many investors cannot or prefer not to short sell, so this constraint reflects real-world limitations. If we remove this constraint, we allow short selling, which can lead to more aggressive portfolios.

Out[5]:
Visualization
Two-dimensional weight space plot showing budget constraint line and elliptical variance contours, with feasible region for portfolio optimization.
Geometric view of portfolio optimization in weight space for two assets. The blue line represents the budget constraint (w₁ + w₂ = 1), while dashed ellipses show iso-variance contours. The optimal minimum-variance portfolio is where the smallest ellipse touches the constraint line. Red shaded regions are infeasible due to no-short-selling constraints.

Mathematical Properties: Why This Problem is Well-Behaved

The quadratic programming formulation for portfolio optimization has several important mathematical properties that make it both theoretically sound and practically solvable:

Convexity: The Guarantee of Global Optimality

The objective function wTΣww^T \Sigma w is convex when the covariance matrix Σ\Sigma is positive semi-definite, which is always true for valid covariance matrices. This convexity property guarantees that:

  • Any local minimum is also a global minimum, so we can't get "stuck" in a suboptimal solution
  • The optimization problem is well-behaved and can be solved efficiently
  • The solution is unique (or the set of optimal solutions is convex)

This convexity makes portfolio optimization computationally tractable. Instead of a complex non-convex function with many local minima, we have a smooth, bowl-shaped function with a single optimal solution.

The Quadratic Nature: Capturing Non-Linear Risk

The quadratic form wTΣww^T \Sigma w captures the essential non-linearity of portfolio risk. Unlike expected returns (which combine linearly), risk grows quadratically with position sizes. This means:

  • Doubling your position in an asset quadruples its contribution to portfolio variance
  • The cross-terms (covariances) create complex interactions that linear models cannot represent
  • Diversification benefits emerge naturally from the mathematical structure

This quadratic relationship is why simple linear approaches fail for portfolio optimization. The problem fundamentally requires a quadratic framework to capture the true nature of portfolio risk.