Liquidity Risk and Other Non-Market Risks › Model Risk

Model risk is the risk of loss arising from decisions based on incorrect or misused models. In quantitative finance, where models drive trading, hedging, pricing, and risk management decisions, model risk is both pervasive and potentially catastrophic.

Models occupy a peculiar position in finance. They are indispensable tools for understanding complex instruments and market dynamics, yet they are also systematic sources of error. Every model simplifies reality, and every simplification carries the risk that some omitted factor will prove important. The danger is magnified by the confidence that quantitative methods can inspire: a precise numerical output may appear more reliable than it is, leading you to underestimate the uncertainty inherent in model-based analysis.

Model risk emerges from several distinct failure modes:

Incorrect model specification occurs when the model's mathematical structure fails to capture the true data-generating process. The Black-Scholes model assumes constant volatility, but as we discussed in the chapter on implied volatility and the volatility smile, this assumption is violated empirically. Using Black-Scholes for exotic options or during volatility regimes different from those assumed can produce systematically biased prices.

The challenge with specification error is that it can be invisible in normal times yet catastrophic in stress. A model calibrated to benign market conditions may fit historical data well, pass backtests, and produce reasonable-looking outputs for an extended period. The flaws only become apparent when conditions shift beyond the model's domain of validity. By then, positions may have been sized based on the model's optimistic risk estimates, and the true risks may be far larger than believed.

Calibration errors arise when model parameters are estimated incorrectly or from inappropriate data. A volatility estimate calibrated to a calm period will understate risk during turbulent markets. The Gaussian copula models used for CDO pricing, which we covered in the chapter on structured credit products, were often calibrated to short historical periods that did not include credit stress, leading to dramatic underestimation of correlation risk.

Calibration error highlights the importance of the data used to estimate model parameters. Parameters estimated from two years of data may not reflect the dynamics that emerge over longer cycles. Parameters estimated from one market regime may not apply in another. The model may be mathematically correct but rendered dangerous by parameters that do not reflect current or future conditions.

Implementation errors occur when the mathematical model is translated incorrectly into code. A misplaced decimal point, incorrect sign, or boundary condition error can cause a correctly specified model to produce wrong outputs. Such bugs may go undetected for extended periods if results appear plausible.

Implementation errors are insidious because they can produce outputs that look reasonable superficially. A pricing model with a sign error might produce prices that are in the right general range, pass cursory sanity checks, and only reveal the problem through systematic P&L discrepancies that accumulate over time. The more complex the model, the more opportunities for implementation error and the harder the error is to detect.

Inappropriate application happens when a model designed for one purpose is used for another. A model calibrated for liquid exchange-traded options may perform poorly for illiquid over-the-counter structures. Using overnight VaR for positions that take weeks to liquidate understates risk.

Every model has a domain of applicability: the conditions under which its assumptions approximately hold and its outputs can be trusted. Using a model outside its domain is a form of model risk that arises from your behavior rather than model construction. It often reflects organizational pressures: a model exists and produces a number, so the number gets used even when the model is not appropriate for the situation at hand.

The London Whale episode at JPMorgan Chase in 2012 illustrates how model risk can enable massive losses. The bank's Chief Investment Office had accumulated large positions in credit derivatives. When the positions moved against the bank, the risk models used by the CIO, which differed from those used by the bank's central risk function, showed lower VaR than alternative approaches would have indicated.

The CIO's VaR model was modified mid-crisis in ways that reduced reported risk, masking the true exposure. By the time the positions were unwound, losses exceeded $6 billion. The incident revealed failures in model governance, approval processes, and the independence of model validation from business units that benefited from favorable model outputs.

The London Whale case illustrates several model risk pathologies. The business unit had latitude to choose its own risk model, creating an incentive to select or modify the model to produce favorable numbers. Model changes were made during a period of stress without adequate scrutiny. The disconnect between the CIO's reported risk and the central risk function's assessment was not escalated effectively. Each of these failures represents a breakdown in model risk governance that allowed the model to become a tool for concealing rather than revealing risk.

Effective model risk management requires multiple layers of defense:

Independent model validation ensures models are reviewed by teams separate from those who build or use them. Validators test model assumptions, check implementations, and assess model performance against out-of-sample data.

The independence of validation matters because those who build or use models have incentives that may not align with accurate risk assessment. Traders want models that support their positions. Structurers want models that help sell products. Risk managers who built a model may be reluctant to acknowledge its flaws. Independent validators, with no stake in the model's outputs, can provide objective assessment.

Model inventory and tiering documents all models in use and classifies them by materiality. Higher-tier models receive more intensive validation and oversight.

Not all models warrant the same attention. A model used to price a single small position should not require the same validation effort as a model used to price a large and complex portfolio. Tiering ensures that validation resources are allocated according to the potential impact of model failure.

Ongoing monitoring tracks model performance over time. When model predictions diverge systematically from realized outcomes, the model may require recalibration or replacement.

Models can degrade over time even without any change to their structure. Market dynamics evolve, parameter estimates become stale, and relationships that held historically may break down. Ongoing monitoring detects this degradation before it causes significant losses.

Limitations documentation explicitly records known model limitations and the conditions under which the model may perform poorly. You must understand these limitations before relying on model outputs.

Every model has limitations, and hiding those limitations creates risk. Explicit documentation forces acknowledgment of what the model cannot do and alerts users to conditions where extra caution is warranted.