Linear Algebra for Quantitative Finance › Systems of Linear Equations

Many financial problems involve systems of linear equations. Hedge construction, factor-exposure matching, and related tasks can often be formulated as $\mathbf{A}\mathbf{x} = \mathbf{b}$ for unknown $\mathbf{x}$, although practical versions may add constraints, approximation, or optimization.

Why is this formulation so ubiquitous? Because linear systems capture the essence of constraints and requirements. In finance, we often face situations where multiple conditions must hold simultaneously. A hedge must neutralize exposure to several risk factors at once, a replicating portfolio must match the payoffs of a target in multiple scenarios, and factor loadings must explain returns across many time periods. Each condition contributes one equation, and the unknowns are the positions or weights we need to determine. Linear algebra provides systematic machinery for finding solutions when they exist and for characterizing what's possible when they don't.

A system of linear equations has the form:

$$\begin{aligned} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n &= b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n &= b_2 \\ &\vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n &= b_m \end{aligned}$$

where:

• $a_{ij}$: the coefficient in equation $i$ for unknown $x_j$
• $x_j$: the $j$-th unknown variable we're solving for
• $b_i$: the right-hand side constant of equation $i$
• $m$: the number of equations
• $n$: the number of unknowns

In matrix form: $\mathbf{A}\mathbf{x} = \mathbf{b}$, where $\mathbf{A}$ is $m \times n$, $\mathbf{x}$ is $n \times 1$, and $\mathbf{b}$ is $m \times 1$.

The matrix $\mathbf{A}$ encodes the structure of the problem: how each unknown contributes to each equation. Finding $\mathbf{x}$ means finding the combination of unknowns that simultaneously satisfies all constraints. In financial applications, $\mathbf{A}$ often represents sensitivities (like Greeks or factor exposures), $\mathbf{x}$ represents positions or weights we're solving for, and $\mathbf{b}$ represents target values we want to achieve.

The geometry of linear systems provides useful intuition. Each equation with at least one nonzero coefficient defines a hyperplane in the space of unknowns (a line in 2D, a plane in 3D, and so on). Solving the system means finding the point (or points, or nothing at all) where all these hyperplanes intersect. When there are exactly as many independent equations as unknowns, and the equations aren't contradictory, the hyperplanes intersect at a single point, giving the unique solution.

When $\mathbf{A}$ is square ($n \times n$) and invertible, the solution is $\mathbf{x} = \mathbf{A}^{-1}\mathbf{b}$. The inverse $\mathbf{A}^{-1}$ satisfies $\mathbf{A}\mathbf{A}^{-1} = \mathbf{A}^{-1}\mathbf{A} = \mathbf{I}$.

The inverse matrix "undoes" the transformation represented by $\mathbf{A}$. If $\mathbf{A}$ transforms inputs to outputs, then $\mathbf{A}^{-1}$ transforms outputs back to inputs. In the context of our linear system, we know the outputs ($\mathbf{b}$, the targets we want to achieve) and need to find the inputs ($\mathbf{x}$, the positions that achieve those targets). Multiplying by the inverse reverses the process, revealing the required inputs.

In numerical work, solve $\mathbf{A}\mathbf{x}=\mathbf{b}$ directly rather than forming $\mathbf{A}^{-1}$ explicitly. A solver can use an appropriate factorization without forming the full inverse. For a single right-hand side, explicitly forming the inverse generally adds computation and can introduce additional rounding error. Rank and condition-number checks are still necessary when the system may be singular or ill-conditioned.

Consider replicating a target portfolio's factor exposures using available assets. Suppose you have three assets with known exposures to two factors (market and size), and you want to construct a portfolio with specific target exposures:

The solution tells us exactly how to combine the three assets to achieve our target factor profile: market beta of 1 with zero size exposure, while being net fully invested. Here the weights are -200%, -200%, and +500%, so the exact match requires short positions and 900% gross exposure. In practice, leverage or short-sale limits can make the exact system infeasible; constrained optimization or regularized least squares can then trade replication error against implementability.

In derivatives trading, you often need to hedge exposure to multiple risk factors. Suppose you hold a portfolio of options and want to eliminate its current modeled sensitivity to the underlying price (delta) and volatility (vega). You have two hedging instruments available. Here delta is the change in dollar value per one-dollar underlying-price change, and vega is the change in dollar value per one-percentage-point implied-volatility change. The option column refers to one modeled option unit, not an exchange contract; portfolio and instrument sensitivities use the same units. Actual contract multipliers must be applied before constructing a hedge matrix.

In this illustrative calculation, shorting 160 shares of stock and buying 20 modeled option units neutralizes both delta and vega at the stated sensitivities. Other risks and later changes in those sensitivities remain.

When we have more equations than unknowns ($m > n$), the system is overdetermined and typically has no exact solution. This happens often in finance. We have many data points (days of returns) but few parameters to estimate (factor exposures).

The least squares solution minimizes the sum of squared residuals:

$$\hat{\mathbf{x}} = \arg\min_{\mathbf{x}} \|\mathbf{b} - \mathbf{A}\mathbf{x}\|_2^2$$

where:

• $\hat{\mathbf{x}}$: the least squares estimate of $\mathbf{x}$
• $\|\cdot\|_2^2$: the squared L2 norm (sum of squared elements)
• $\mathbf{b} - \mathbf{A}\mathbf{x}$: the residual vector (observed minus fitted)

The solution is given by the normal equations (assuming $\mathbf{A}^T\mathbf{A}$ is invertible):

$$\hat{\mathbf{x}} = (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{b}$$

where:

• $\mathbf{A}^T\mathbf{A}$: the $n \times n$ Gram matrix of the explanatory-variable columns, whose entries are their pairwise inner products
• $\mathbf{A}^T\mathbf{b}$: a vector containing each explanatory-variable column's inner product with the target vector

Differentiating the squared residual norm gives the first-order condition $\mathbf{A}^T(\mathbf{A}\hat{\mathbf{x}}-\mathbf{b})=\mathbf{0}$, or $\mathbf{A}^T\mathbf{A}\hat{\mathbf{x}}=\mathbf{A}^T\mathbf{b}$. When the Gram matrix is invertible, solving these normal equations yields the displayed formula. The Gram matrix captures column scale and non-orthogonality; it is not generally a correlation matrix, and its inverse is not a literal correction for "double-counting."

When $\mathbf{A}^T\mathbf{A}$ is invertible, the matrix $(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T$ is the Moore-Penrose pseudoinverse of $\mathbf{A}$.

The formula describes the mathematical solution, not a recommended floating-point algorithm. For a full-column-rank matrix, forming $\mathbf{A}^T\mathbf{A}$ squares its 2-norm condition number and can obscure weak directions numerically. Prefer a QR- or SVD-based least-squares algorithm. The example below calls np.linalg.lstsq directly rather than manually forming and inverting the Gram matrix.

Intuitively, least squares finds the $\mathbf{x}$ that makes $\mathbf{A}\mathbf{x}$ as close as possible to $\mathbf{b}$ in Euclidean distance. The residual vector $\mathbf{b} - \mathbf{A}\hat{\mathbf{x}}$ is orthogonal to the column space of $\mathbf{A}$. It is the component not represented by the chosen explanatory-variable columns; changing those columns or the model form can change the residual.

This geometric picture explains why least squares produces the best fit within the chosen column space. Among all possible values of $\mathbf{x}$, the least squares solution generates an $\mathbf{A}\mathbf{x}$ that is the orthogonal projection of $\mathbf{b}$ onto the subspace spanned by the columns of $\mathbf{A}$. Orthogonal projection onto a subspace always yields the closest point in that subspace. The residual is therefore the in-sample component unexplained by this span. It may contain noise, measurement error, omitted predictors, or misspecification; the projection geometry alone does not identify irreducible statistical error.

This is exactly what ordinary least-squares linear regression computes.