Linear Algebra for Quantitative Finance › Matrix Decompositions

Matrix decompositions break a matrix into simpler components, exposing structure that's hidden in the raw numbers. In quantitative finance, decompositions help us study risk directions, enable dimensionality reduction through truncation, and improve numerical stability.

Think of decomposition as a kind of mathematical X-ray. A covariance matrix appears as a dense array of numbers, but its eigendecomposition reveals modes of variation, the basic ways in which assets tend to move together. A return matrix might contain hundreds of thousands of numbers, but its singular value decomposition exposes dominant patterns in the observed variation. These mathematical components can support financial interpretation after their loadings, scores, and stability have been examined; the decomposition itself does not assign economic meaning.

A diagonalizable square matrix $\mathbf{A}$ has a complete eigenvector basis and can be expressed as:

where:

• $\mathbf{V}$: an $n \times n$ matrix whose columns are the eigenvectors $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$
• $\mathbf{\Lambda}$: a diagonal matrix with eigenvalues $\lambda_1, \lambda_2, \ldots, \lambda_n$ on the diagonal
• $\mathbf{V}^{-1}$: the inverse of the eigenvector matrix, which "undoes" the coordinate transformation

This decomposition shows that $\mathbf{A}$ acts as: (1) changing into the eigenvector coordinate system via $\mathbf{V}^{-1}$, (2) scaling each coordinate by the corresponding eigenvalue via $\mathbf{\Lambda}$, and (3) changing back via $\mathbf{V}$. These changes of basis are rotations or reflections only when $\mathbf{V}$ is orthogonal, as it can be for a real symmetric covariance matrix. In that covariance setting, the decomposition separates uncorrelated directions of variation.

The power of this decomposition lies in the simplicity of the diagonal matrix $\mathbf{\Lambda}$. A diagonal matrix just scales each coordinate independently, with no mixing between directions. All the complexity of $\mathbf{A}$ is absorbed into finding the right coordinate system (the eigenvectors) in which the matrix action becomes simple scaling.

For a covariance matrix, the eigenvectors represent the principal directions of variation in the data, and the eigenvalues represent the variance along each direction. Among unit-length directions $\mathbf{v}$, the leading eigenvector maximizes $\mathbf{v}^T\Sigma\mathbf{v}$ and the maximum equals the largest eigenvalue. This describes the highest-variance direction in the asset universe; the contribution to a particular portfolio's variance also depends on that portfolio's weights.

PCA uses eigendecomposition to transform correlated variables into uncorrelated principal components. The full projection preserves the original dimension; dimensionality reduction occurs only when we retain $k<N$ leading components. Whether a small number explains most variation depends on the assets, sample period, return frequency, scaling, and preprocessing.

The principal components are projections of the data onto the eigenvectors:

where:

• $\mathbf{Z}$: the full $T \times N$ matrix of principal component scores; retaining its first $k<N$ columns gives a reduced $T \times k$ representation
• $\mathbf{X}$: the $T \times N$ centered data matrix (each row is an observation, each column is a variable minus its mean)
• $\mathbf{V}$: the $N \times N$ matrix of eigenvectors (principal component loadings)

Each column of $\mathbf{Z}$ represents a principal component, a new synthetic variable that captures a specific pattern of co-movement in the original data. The first principal component captures the most variance, the second captures the most remaining variance while being uncorrelated with the first, and so on. Analysts may assign economic labels such as market, sector, or style factors after inspecting the loadings and scores, but PCA does not guarantee that a component has a clean economic interpretation.

Why does this step produce uncorrelated components? Let $\Sigma$ be the covariance matrix of the centered data and let the columns of $\mathbf{V}$ be its orthonormal eigenvectors. Then

Because $\mathbf{\Lambda}$ is diagonal, distinct principal-component scores have zero sample covariance. Orthogonal axes alone would not guarantee this for arbitrary data; the result holds because PCA chooses axes that diagonalize $\Sigma$. The eigenvalues give the variances along those axes, so ordering them places the main variance direction first.

In this seeded sample, the first principal component has same-sign loadings and can be read as a broad co-movement direction. The later components have mixed signs and contrast assets. These interpretations are sample-dependent, and multiplying any loading vector by -1 describes the same principal axis, so loading signs are a convention rather than an economic direction by themselves.

PCA is often applied to yield-curve movements because rates at different maturities tend to co-move. In the synthetic example below, the first three components explain 92.3% of sample variation and their loading shapes resemble three commonly used descriptions:

1. Level (PC1): A broad same-direction shift resembling a level change; loading magnitudes need not be equal across maturities.
2. Slope (PC2): Steepening or flattening. Short and long rates move in opposite directions.
3. Curvature (PC3): Bending. The middle of the curve moves relative to the ends.

Retaining the first three components below produces a $500 \times 3$ score matrix instead of the original $500 \times 8$ data matrix. This reduces the dimension, but the variance retained and the economic labels depend on the sample and preprocessing. A fixed-income risk analysis can track these component exposures alongside maturity-specific risks that the reduced representation leaves out.

The synthetic daily yield changes and their marginal standard deviations are measured in percentage points: 0.05 percentage points is 5 basis points. This fixes the financial scale without changing the PCA loading directions or explained-variance ratios.

In this synthetic sample, PC1 has same-sign loadings across maturities and resembles a level shift. PC2 contrasts short and long maturities, while PC3 contrasts the middle with the ends and resembles curvature. The signs could all be reversed without changing the components. These loading patterns can be used to describe and hedge component exposures, subject to the fit and stability of the estimated PCA model.

While eigendecomposition only applies to square matrices, Singular Value Decomposition works for any matrix. For an $m \times n$ matrix $\mathbf{A}$:

where:

• $\mathbf{U}$: an $m \times m$ orthogonal matrix whose columns are the left singular vectors. The vectors associated with non-zero singular values span the column space of $\mathbf{A}$; for a time-by-asset return matrix, their coordinates run across observations such as days.
• $\mathbf{\Sigma}$: an $m \times n$ diagonal matrix with non-negative singular values $\sigma_1 \geq \sigma_2 \geq \cdots \geq 0$ on the diagonal
• $\mathbf{V}$: an $n \times n$ orthogonal matrix whose columns are the right singular vectors. The vectors associated with non-zero singular values span the row space of $\mathbf{A}$; for a time-by-asset return matrix, their coordinates run across assets.

The code below requests NumPy's reduced SVD. With $k=\min(m,n)$, full_matrices=False returns U with shape $m \times k$, singular_values as a length-$k$ vector, and Vt with shape $k \times n$. NumPy returns the diagonal entries rather than an explicit $\mathbf{\Sigma}$ matrix; U @ np.diag(singular_values) @ Vt reconstructs $\mathbf{A}$.

The singular values measure the "importance" of each component. For a return matrix with days as rows and assets as columns, $\mathbf{U}$ shows when certain patterns occurred, $\mathbf{V}$ shows which assets participated in each pattern, and $\Sigma$ shows how strong each pattern was.

The strength of SVD lies in its universality and interpretability. Any real matrix, not just square symmetric ones, can be decomposed into orthogonal changes of basis, including rotations or reflections, and a rectangular diagonal map with nonnegative scalings. The singular values are always non-negative and ordered by magnitude, and they provide a natural ranking by component strength. Truncating the SVD by keeping only the largest singular values gives the best low-rank approximation to the original matrix in terms of Frobenius norm, a result known as the Eckart-Young theorem. This makes SVD the mathematical foundation for dimensionality reduction and data compression.

SVD is useful for handling non-square return matrices (different numbers of days vs. assets) and for computing the pseudoinverse used in least squares solutions.

The close match between eigenvalues and σ²/(T-1) confirms the mathematical relationship between SVD and eigendecomposition. The largest singular value corresponds to the dominant variance pattern in the return matrix. That pattern may be interpreted as a market factor only after inspecting and validating its asset loadings and time scores; SVD alone does not assign an economic label. This relationship is why PCA can be computed efficiently via SVD, which is numerically more stable than computing the eigendecomposition of the covariance matrix directly.

For positive definite matrices, including covariance matrices that are strictly positive definite, Cholesky decomposition provides a "square root":

where:

• $\mathbf{L}$: a lower triangular matrix (all entries above the diagonal are zero)
• $\mathbf{L}^T$: the transpose of $\mathbf{L}$ (an upper triangular matrix)

For a dense $n \times n$ matrix, Cholesky factorization requires about $n^3/3$ floating-point operations. Dense LU factorization requires about $2n^3/3$, so both are $O(n^3)$ but Cholesky has roughly half the leading operation count when its positive-definite assumptions hold.

When the diagonal entries of $\mathbf{L}$ are required to be positive, this lower-triangular Cholesky factor is unique. The factor can be applied efficiently to independent draws to simulate correlated variables, a common step in Monte Carlo derivatives-pricing and risk workflows.

The lower triangular structure of $\mathbf{L}$ gives a sequential construction relative to the chosen variable ordering. The first simulated component uses one independent shock, the second mixes that shock with a second one, and the third mixes the first two shocks with a third. This is a computational representation of the target covariance, not a causal claim that later-listed assets depend on earlier-listed assets. Reordering the assets generally changes the factor $\mathbf{L}$ while preserving the appropriately reordered covariance matrix; permutation-invariant cases, such as the identity matrix, can leave the factor unchanged.