Maximum Likelihood Estimation › Formula

We can build the likelihood in three steps. First, hold the observed data fixed; they are no longer random quantities to be predicted. Second, choose a probability model that assigns a probability or density to each observation for any candidate parameter value. Third, compare candidate parameter values by asking which one gives the entire observed sample the highest combined plausibility. The formula below is simply the bookkeeping for those three steps.

Suppose we observe independent and identically distributed data $y_1,\dots,y_n$. Let $f(y_i;\theta)$ be the probability mass function for discrete data or the probability density function for continuous data, where $\theta$ is a parameter or a vector of parameters. The likelihood is

$$L(\theta; y_1,\dots,y_n) = \prod_{i=1}^{n} f(y_i;\theta).$$

where:

• $L(\theta; y_1,\dots,y_n)$: the likelihood of the parameter $\theta$ given the full observed sample
• $n$: the number of observations in the sample
• $y_i$: the $i$-th observed data value, held fixed as $\theta$ varies
• $f(y_i;\theta)$: the probability mass (discrete case) or probability density (continuous case) assigned to $y_i$ by the model with parameter $\theta$
• $\prod_{i=1}^{n}$: the product over all observations from $i=1$ to $i=n$

The product symbol $\prod_{i=1}^{n}$ means that we multiply the terms for $i=1$ through $i=n$. That multiplication is not an arbitrary convention. Independence is exactly what makes the joint probability or joint density of the sample factor into one term per observation. If the observations were dependent, the product of marginal terms would not be the joint distribution, and the likelihood would have to be built from the joint model instead. The semicolon in $f(y_i;\theta)$ separates the observed data $y_i$ from the parameter $\theta$; it reminds us that the data are fixed and $\theta$ is the input we are varying. For discrete data, each $f(y_i;\theta)$ is a probability, so the product is the probability of the observed sequence under independence. For continuous data, each term is a density, so the product is a likelihood density rather than the probability of observing an exact value.

The log-likelihood is

$$\begin{aligned} \ell(\theta) &= \log L(\theta; y_1,\dots,y_n) \\ &= \log \prod_{i=1}^{n} f(y_i;\theta) && \text{(substitute the likelihood)} \\ &= \sum_{i=1}^{n} \log f(y_i;\theta) && \text{(log of a product is a sum)}. \end{aligned}$$

The symbols in this expression are:

• $\ell(\theta)$: the log-likelihood, the natural logarithm of the likelihood
• $\log$: the natural logarithm
• $\sum_{i=1}^{n}$: the sum over all observations from $i=1$ to $i=n$
• the remaining symbols are defined as above.

The logarithm is monotone increasing. If $L(\theta_1) > L(\theta_2)$, then $\ell(\theta_1) > \ell(\theta_2)$. Therefore the parameter that maximizes $L$ also maximizes $\ell$. Logs turn products into sums, which are easier to differentiate and much less likely to underflow when $n$ is large. It is common to see negative log-likelihood minimized instead, because optimization software usually minimizes rather than maximizes. These three facts, factorization under independence, monotone transformation, and numerical stability, explain nearly every manipulation that follows.

The likelihood is a function of the parameter, not a probability distribution over the parameter. This point deserves emphasis. If we compute $L(0.3)=0.002$ and $L(0.6)=0.010$, we can say that the observed data are five times more likely under $0.6$ than under $0.3$ in the likelihood ratio sense. We cannot say that there is a 1% probability that the parameter equals $0.6$. To obtain a probability distribution over $\theta$, we would need a prior distribution and a normalization step, which leads to Bayesian inference. Here we remain in the frequentist likelihood framework.

The independence assumption is what allows us to multiply the individual terms. If observations are correlated, the product of marginal probabilities is not the joint probability, and the likelihood must be built from the joint distribution instead. Identically distributed means that every observation has the same distribution. Independent observations may instead have different densities, all indexed by a shared parameter; their likelihood is still the product of those individual densities. Dependence requires a joint model.

For discrete data, the likelihood is a probability of the observed data. For continuous data, it is a density. A density can be greater than one when the distribution is concentrated over a narrow range, so a likelihood value above one is not an error. For estimation, what matters is the relative size of the likelihood at different parameter values, not whether the number looks like a probability.

To turn this function into an estimate, we need a way to locate its peak. The score is the first derivative of the log-likelihood:

$$s(\theta) = \frac{\partial \ell(\theta)}{\partial \theta}.$$

where:

• $s(\theta)$: the score, the first derivative of the log-likelihood with respect to the parameter
• $\theta$: the parameter (or parameter vector) being estimated
• $\partial \ell(\theta)/\partial \theta$: the partial derivative of the log-likelihood with respect to $\theta$

At a differentiable maximum in the interior of the parameter space, the score is zero:

$$s(\hat\theta)=0.$$

This equation is called the score equation or the likelihood equation. In simple models we can solve it algebraically. In more complex models we solve it numerically. The score tells us the local slope of the log-likelihood. A positive score means increasing $\theta$ increases the log-likelihood, and a negative score means decreasing $\theta$ increases it. A zero score means we are at a flat point, which could be a maximum, a minimum, or a saddle point. A negative second derivative gives a strict local maximum in one dimension; a positive one gives a strict local minimum. A zero second derivative is inconclusive. For a vector parameter, negative definiteness of the Hessian is a sufficient local maximum test.

The general formula becomes concrete only after we choose a probability model. You now derive two estimators that appear constantly in practice. The first is the Bernoulli probability, which gives us the sample proportion. The second is the Gaussian mean and variance, which gives us the sample mean and the maximum likelihood variance. These derivations show how the general likelihood formula becomes a concrete estimate.

For Bernoulli data, each observation is either $1$ or $0$. Assume $n>0$, and let $p\in[0,1]$ be the probability of a success. For $0<p<1$, the probability mass is

$$f(y_i;p)=p^{y_i}(1-p)^{1-y_i}.$$

where:

• $y_i$: the $i$-th observation, taking the value $1$ for a success or $0$ for a failure
• $p$: the probability of a success, the unknown parameter
• $1-p$: the probability of a failure

When $y_i=1$, this becomes $p^1(1-p)^0=p$. When $y_i=0$, it becomes $p^0(1-p)^1=1-p$. If we observe $k$ successes and $n-k$ failures, then the product over all observations is

$$L(p)=p^{k}(1-p)^{n-k}.$$

where:

• $k$: the number of observed successes
• $n-k$: the number of observed failures
• $p$: the success probability

The log-likelihood is

$$\ell(p)=k\log p+(n-k)\log(1-p).$$

where:

• $\ell(p)$: the log-likelihood for the Bernoulli success probability $p$
• $k$: the number of observed successes
• $n-k$: the number of observed failures
• $p$: the success probability

Differentiating with respect to $p$ gives

$$\frac{d\ell}{dp}=\frac{k}{p}-\frac{n-k}{1-p}.$$

where:

• $\frac{d\ell}{dp}$: the derivative of the Bernoulli log-likelihood with respect to $p$
• $\frac{k}{p}$: the term contributed by the successes
• $-\frac{n-k}{1-p}$: the derivative term contributed by the failures

Setting this equal to zero, we multiply both sides by $p(1-p)$:

$$k(1-p)-p(n-k)=0.$$

where:

• $k$: the number of observed successes
• $1-p$: the candidate failure probability
• $n-k$: the number of observed failures
• $p$: the success probability

Expanding gives $k-kp-pn+pk=0$, and the $-kp+pk$ terms cancel, leaving $k-pn=0$. Therefore

$$\hat p=\frac{k}{n}.$$

The second derivative is

$$\frac{d^2\ell}{dp^2}=-\frac{k}{p^2}-\frac{n-k}{(1-p)^2}<0,$$

where:

• $\frac{d^2\ell}{dp^2}$: the second derivative of the Bernoulli log-likelihood
• $\frac{k}{p^2}$: the magnitude of the negative curvature contribution from the successes
• $\frac{n-k}{(1-p)^2}$: the magnitude of the negative curvature contribution from the failures

This derivative is negative for $0<p<1$. With $0<k<n$, the stationary point is the unique global maximum. On the closed parameter space $[0,1]$, $k=0$ or $k=n$ puts the maximum at $p=0$ or $p=1$, using the limiting convention $0\log 0=0$. The plug-in Wald standard error is zero there, but that does not establish zero uncertainty. The interior information formula below is not valid at those endpoints.

If we only record the count $k$ rather than the exact sequence of successes and failures, the binomial likelihood is $\binom{n}{k}p^k(1-p)^{n-k}$. The binomial coefficient does not depend on $p$, so it does not change the maximizing value. That is why we can work with the simpler Bernoulli form.

For Gaussian data, suppose $y_i\sim N(\mu,\sigma^2)$. The density is

$$f(y_i;\mu,\sigma^2)=\frac{1}{\sqrt{2\pi\sigma^2}} \exp\left[-\frac{(y_i-\mu)^2}{2\sigma^2}\right].$$

where:

• $\mu$: the mean of the normal distribution
• $\sigma^2$: the variance of the normal distribution
• $\sigma$: the standard deviation, the positive square root of the variance
• $\exp[\cdot]$: the exponential function

The log-likelihood for all $n$ observations is

$$\ell(\mu,\sigma^2) = -\frac{n}{2}\log(2\pi) -\frac{n}{2}\log(\sigma^2) -\frac{1}{2\sigma^2}\sum_{i=1}^{n}(y_i-\mu)^2.$$

where:

• $\ell(\mu,\sigma^2)$: the Gaussian log-likelihood for all $n$ observations
• $n$: the number of observations
• $\log(2\pi)$: the constant log term from the normalizing factor
• $\log(\sigma^2)$: the log of the variance
• $\frac{1}{2\sigma^2}\sum_{i=1}^{n}(y_i-\mu)^2$: the sum of squared deviations divided by twice the variance

To estimate $\mu$, we differentiate with respect to $\mu$. The first two terms do not depend on $\mu$, and the derivative of the squared term brings down a factor of $-2(y_i-\mu)$:

$$\frac{\partial \ell}{\partial \mu} = \frac{1}{\sigma^2}\sum_{i=1}^{n}(y_i-\mu).$$

where:

• $\frac{\partial \ell}{\partial \mu}$: the score with respect to the Gaussian mean
• $\sigma^2$: the variance of the normal distribution
• $y_i-\mu$: the deviation of the $i$-th observation from the candidate mean

Setting this to zero gives

$$\sum_{i=1}^{n}(y_i-\mu)=0 \quad\Longrightarrow\quad \sum_{i=1}^{n}y_i-n\mu=0 \quad\Longrightarrow\quad \hat\mu=\frac{1}{n}\sum_{i=1}^{n}y_i=\bar y.$$

where:

• $\sum_{i=1}^{n}(y_i-\mu)=0$: the score set to zero
• $\sum_{i=1}^{n}y_i$: the sum of all observations
• $n\mu$: the sum if every observation were equal to $\mu$
• $\hat\mu$: the maximum likelihood estimate of the mean
• $\bar y$: the sample mean

The maximum likelihood estimate of the mean is the sample mean. This result does not depend on the value of $\sigma^2$.

To estimate the variance, let $v=\sigma^2$ and write the sum of squared deviations as

$$\mathrm{SSE}=\sum_{i=1}^{n}(y_i-\bar y)^2.$$

After substituting $\mu=\bar y$, the profile log-likelihood for $v>0$ becomes

$$\ell(\bar y,v) = -\frac{n}{2}\log(2\pi) -\frac{n}{2}\log v -\frac{\mathrm{SSE}}{2v}.$$

where:

• $\ell(\bar y,v)$: the Gaussian log-likelihood with the mean fixed at its fitted value and candidate variance $v$
• $n$: the number of observations
• $v$: the candidate variance $\sigma^2$
• $\mathrm{SSE}$: the sum of squared deviations from the sample mean

Differentiating with respect to $v$ gives

$$\frac{\partial \ell}{\partial v} = -\frac{n}{2v}+\frac{\mathrm{SSE}}{2v^2}.$$

Setting this to zero, we multiply by $2v^2$:

$$-nv+\mathrm{SSE}=0 \quad\Longrightarrow\quad \hat v=\frac{\mathrm{SSE}}{n}.$$

where:

• $n$: the number of observations
• $v$: the candidate variance
• $\mathrm{SSE}$: the sum of squared deviations from the sample mean
• $\hat v$: the maximum likelihood variance estimate

If $\mathrm{SSE}>0$, the profile likelihood rises for $v<\mathrm{SSE}/n$ and falls for $v>\mathrm{SSE}/n$, so this is the global maximum over $v>0$. If all observations are identical, $\mathrm{SSE}=0$ and the Gaussian likelihood is unbounded as $v$ tends to zero: no positive-variance MLE is attained. For nondegenerate i.i.d. Gaussian samples, the variance estimator is biased downward because

$$\mathbb{E}\left[\frac{\mathrm{SSE}}{n}\right] =\frac{n-1}{n}\sigma^2.$$

where:

• $\mathbb{E}[\cdot]$: expectation over repeated samples
• $\mathrm{SSE}$: the sum of squared deviations from the sample mean
• $n$: the number of observations
• $\sigma^2$: the true variance

The bias shrinks as $n$ grows, and the estimator is consistent, but it is not unbiased. This is a concrete reminder that maximum likelihood and unbiasedness are different properties.

The Gaussian likelihood also explains why squared error appears so often in regression. If we model $y=X\beta+\varepsilon$ with $\varepsilon\sim N(0,\sigma^2 I)$, the log-likelihood is

$$\ell(\beta,\sigma^2) = -\frac{n}{2}\log(2\pi\sigma^2) -\frac{1}{2\sigma^2}(y-X\beta)^\top(y-X\beta).$$

Here $y$ is a column vector of responses, $X$ is a matrix with one row per observation and one column per predictor, $\beta$ is a column vector of coefficients, and $I$ is the identity matrix. The expression $(y-X\beta)^\top(y-X\beta)$ is exactly the sum of squared errors written in matrix notation. For a fixed $\sigma^2$, maximizing the likelihood over $\beta$ is the same as minimizing the sum of squared errors. We will see the full regression derivation in the later chapters on ordinary least squares, but the likelihood connection is already visible here.

The mean controls the center of the distribution, and the standard deviation controls its width. When you evaluate a likelihood, you are not plotting a distribution over $\mu$ or $\sigma$. You are comparing candidate distributions over the observed data $y$.

The score equation $s(\hat\theta)=0$ is the first-order condition for a maximum. To confirm a maximum and to measure uncertainty, we use the second derivative. Intuitively, the first derivative tells us where the likelihood is flat, while the second derivative tells us how quickly it curves away from that flat point. A sharp peak is informative. A broad, nearly flat peak is not. The observed information is

$$J(\hat\theta)=-\left.\frac{d^2\ell(\theta)}{d\theta^2}\right|_{\theta=\hat\theta}.$$

where:

• $J(\hat\theta)$: the observed information, evaluated at the estimate $\hat\theta$
• $\hat\theta$: the maximum likelihood estimate of the parameter
• $d^2\ell(\theta)/d\theta^2$: the second derivative of the log-likelihood with respect to $\theta$
• the vertical bar with subscript $\theta=\hat\theta$: evaluation of the second derivative at the estimate

For a vector parameter, the second derivative becomes a Hessian matrix, and the observed information is the negative Hessian:

$$J(\hat\theta)=-\nabla^2\ell(\hat\theta).$$

The Fisher information is the expected value of the observed information under the model:

$$I_n(\theta) = \mathbb{E}_{\theta}\left[J(\theta)\right] = \mathbb{E}_{\theta}\left[-\nabla^2\ell(\theta)\right],$$

where:

• $I_n(\theta)$: the Fisher information in the full sample for the parameter $\theta$
• $\mathbb{E}_{\theta}[\cdot]$: the expectation taken over repeated samples generated with the true parameter equal to $\theta$
• $\nabla^2\ell(\theta)$: the Hessian of the log-likelihood (the matrix of second partial derivatives)

For a correctly specified regular i.i.d. model, $I_n(\theta)=nI_1(\theta)$, where $I_1$ is per-observation information. Under the usual consistency and derivative-convergence conditions, $J(\hat\theta)/n$ and $I_n(\hat\theta)/n$ converge to the same nonsingular limit. This statement concerns information per observation; their unnormalized difference need not tend to zero.

At a regular interior estimate with positive information, a large-sample standard error for a scalar parameter is

$$\widehat{\mathrm{se}}(\hat\theta) \approx \sqrt{J(\hat\theta)^{-1}} = \sqrt{\frac{1}{-\ell''(\hat\theta)}}.$$

where:

• $\widehat{\mathrm{se}}(\hat\theta)$: the approximate standard error of the estimate
• $J(\hat\theta)^{-1}$: the reciprocal of the observed information (the inverse information matrix in the vector case)
• $\ell''(\hat\theta)$: the second derivative of the log-likelihood evaluated at the estimate

For a regular interior vector estimate with nonsingular positive-definite observed information, its inverse approximates the covariance matrix:

$$\widehat{\mathrm{Cov}}(\hat\theta) \approx \left[-\nabla^2\ell(\hat\theta)\right]^{-1}.$$

The diagonal entries give approximate variances, and their square roots give approximate standard errors. This is where the shape of the likelihood becomes uncertainty: a sharply curved peak has large information and small standard error, while a flat peak has small information and large standard error.

The quadratic approximation below shows why curvature is used as a local summary of information. Near the peak, the log-likelihood and its second-order approximation almost coincide; farther away, the approximation drifts from the exact curve.

For the Bernoulli model with $0<k<n$, the observed information at $\hat p=k/n$ is

$$J(\hat p)=\frac{n}{\hat p(1-\hat p)},$$

where:

• $J(\hat p)$: the observed information for the Bernoulli probability
• $n$: the number of trials
• $\hat p$: the estimated success probability
• $1-\hat p$: the estimated failure probability

so the approximate standard error is

$$\widehat{\mathrm{se}}(\hat p) = \sqrt{\frac{\hat p(1-\hat p)}{n}}.$$

where:

• $\widehat{\mathrm{se}}(\hat p)$: the approximate standard error of the Bernoulli estimate
• $\hat p$: the estimated success probability
• $1-\hat p$: the estimated failure probability
• $n$: the number of trials

This is the familiar standard error of a sample proportion. For the Gaussian mean with known variance, the observed information is $n/\sigma^2$, so the standard error is $\sigma/\sqrt{n}$. For the Gaussian variance, the observed information at $\hat v=\mathrm{SSE}/n$ is $n/(2\hat v^2)$. The approximate standard error is therefore $\hat v\sqrt{2/n}$, where $\hat v=\hat\sigma^2$ is the variance estimate.

When no closed-form solution exists, you can maximize the log-likelihood numerically. A simple one-dimensional approach is a grid search: evaluate the log-likelihood on a fine grid of parameter values and choose the grid point with the largest value. Grid search is easy to reason about but becomes impractical in more than a few dimensions.

A more scalable approach is Newton-Raphson. It comes from a second-order Taylor expansion of the objective around the current iterate, and it approximately solves the score equation $\nabla\ell(\theta)=0$ at each step. To maximize $\ell(\theta)$, you update

$$\theta_{t+1} = \theta_t - \left[\nabla^2\ell(\theta_t)\right]^{-1} \nabla\ell(\theta_t).$$

where:

• $\theta_t$: the parameter value at iteration $t$
• $\theta_{t+1}$: the updated parameter value
• $\nabla\ell(\theta_t)$: the gradient (score) of the log-likelihood at $\theta_t$
• $\nabla^2\ell(\theta_t)$: the Hessian matrix of second derivatives at $\theta_t$
• $[\cdot]^{-1}$: the matrix inverse

The gradient points in the direction of increase, and the inverse Hessian rescales that direction. The update is equivalent to solving $\nabla^2\ell(\theta_t)(\theta_{t+1}-\theta_t) = -\nabla\ell(\theta_t)$, which makes the score approximately zero after the step. When the Hessian is negative definite and the score is nonzero, the Newton direction is an ascent direction, but the full step can overshoot and decrease the likelihood. A line search can shorten the step. If the starting value is poor or the Hessian is not negative definite, Newton-Raphson can diverge. Practical implementations often minimize the negative log-likelihood with quasi-Newton methods, add bounds, or reparameterize positive quantities using logarithms. For example, instead of optimizing $\sigma^2$ directly, you can optimize $\log\sigma$ and exponentiate the result. This keeps the standard deviation positive.

A model is identifiable if different parameter values produce different distributions for the data. Formally, if

$$P_{\theta_1}(A)=P_{\theta_2}(A) \quad\text{for every measurable event }A$$

implies $\theta_1=\theta_2$, then the parameter is identifiable. Here $P_\theta(A)$ is the model probability of an event $A$. If distinct parameters give the same distribution, no amount of data from that model can distinguish those equivalent values. A continuous equivalence can create a flat ridge, while a discrete label symmetry can create separate equivalent peaks. A mixture model with unlabeled components is a classic example: swapping the labels of two components gives the same likelihood. In linear regression, perfectly collinear predictor columns make $X^\top X$ singular, so different coefficient vectors can produce the same fitted values. We discussed the practical consequences of that in Multicollinearity.

Identifiability is a property of the model and the data design, not a property of the optimizer. A numerical optimizer may return a point on a flat ridge, but the data do not distinguish that point from the other equivalent points. Before trusting an estimate, ask whether the data contain enough variation to distinguish the parameters. If not, we need to simplify the model, collect more informative data, or impose a constraint.

Sampling uncertainty is different from identifiability. Even in an identifiable model, the maximum likelihood estimate is a random variable because it depends on a random sample. For a correctly specified identifiable model with an interior true parameter, sufficient smoothness, nonsingular Fisher information, a consistent maximizing solution, and the derivative-convergence conditions used in the asymptotic argument, we obtain:

$$\sqrt{n}\left(\hat\theta-\theta_0\right) \xrightarrow{d} N\left(0,I_1(\theta_0)^{-1}\right),$$

where:

• $\sqrt{n}$: the square root of sample size, which scales the centered estimation error to a nondegenerate limit under the stated regularity conditions
• $\hat\theta$: the maximum likelihood estimate
• $\theta_0$: the true parameter value
• $\xrightarrow{d}$: convergence in distribution as $n\to\infty$
• $N(0,I_1(\theta_0)^{-1})$: the limiting normal distribution with mean zero and covariance $I_1(\theta_0)^{-1}$
• $I_1(\theta_0)$: the per-observation Fisher information evaluated at the true parameter $\theta_0$

This is the basis for the curvature-based standard errors above. The approximation can be poor in small samples, near boundaries, or when the model is misspecified. Alternatives include profile likelihood, a bootstrap appropriate to the sampling design, and Bayesian inference with an explicit prior. Each requires its own assumptions and coverage or sensitivity checks; none automatically repairs a boundary or a misspecified model.

Maximum likelihood also has failure modes that are worth recognizing. The likelihood can be unbounded, as in a Gaussian mixture when a component variance collapses to zero around a single point. In logistic regression, complete separation can push coefficients toward infinity because the likelihood keeps increasing as the separating boundary becomes sharper. Non-concave likelihoods can have multiple local maxima, so different starting values may produce different answers. And if the model is wrong, the estimate may be precise but precisely wrong.

Maximum likelihood has several properties that make it useful beyond the basic examples. The first is invariance under reparameterization. If $\hat\theta$ is the maximum likelihood estimate of $\theta$, and $g$ is a one-to-one function, then $g(\hat\theta)$ is the maximum likelihood estimate of $g(\theta)$. For example, if $0<\hat p<1$, then $\hat p/(1-\hat p)$ is the finite maximum likelihood estimate of the odds. This property is convenient, but it does not mean that standard errors transform by simply plugging in the transformed estimate. Uncertainty usually requires the delta method or a profile-likelihood calculation.

The second property is additivity for independent observations. The log-likelihood is a sum of per-observation log-density or log-mass contributions, so each new independent observation adds one term. This makes the likelihood scalable and explains why the curvature typically grows with sample size. In regular models, information adds across independent observations. For a correctly specified regular i.i.d. model with fixed, finite, positive definite per-observation information, total information is proportional to $n$, and the usual large-sample standard errors shrink at rate $1/\sqrt{n}$.

Under regularity conditions, maximum likelihood estimators are consistent and asymptotically efficient. Consistency means the estimate converges to the true parameter as the sample size grows. Asymptotic efficiency means attaining the inverse-Fisher-information bound within the regular estimator class. It does not compare every possible consistent estimator at every parameter value. These are large-sample guarantees. They do not imply that the estimate is unbiased in a small sample, that the model is correct, or that the likelihood is well behaved in every finite dataset.

Finally, maximum likelihood is not the same as least squares in general. For independent Gaussian errors and any fixed common positive variance, maximizing over the mean parameters is equivalent to minimizing squared error. Bernoulli, Poisson, and other non-Gaussian likelihoods generally give different objectives, although their maximizing estimates can sometimes coincide with least-squares estimates. This is why the likelihood framework is more general than squared error, and why the choice of likelihood encodes assumptions about the data-generating process.