Maximum Likelihood Estimation › Disadvantages

Maximum likelihood inherits every assumption of the probability model. If we choose a Gaussian model for data with heavy outliers, the resulting mean can be pulled strongly toward those outliers. If correlated Bernoulli observations are treated as independent, the standard errors can be wrong. Positive covariance terms increase the variance of the sample proportion; negative terms can decrease it. With a misspecified model, the estimator may converge to a pseudo-true parameter that is closest to the data-generating process in a Kullback-Leibler sense. That parameter may not answer the scientific or business question you care about.

Maximum likelihood is also not guaranteed to be unbiased in finite samples. The Gaussian variance estimator is a standard example: it divides the sum of squared errors by $n$, not $n-1$, so it is biased downward on average. At boundaries, such as a Bernoulli probability estimated from all successes, the maximum may sit at the edge of the parameter space, and the usual normal approximation can break down. Some models are not identifiable, meaning different parameter values produce exactly the same distribution for the data, so no amount of data can separate them.

Numerical difficulties are common as well. The likelihood can be flat, multimodal, or unbounded in unconstrained parameterizations. Optimization may stop at a local maximum, fail because of scaling, or wander toward infinity when a parameter approaches a boundary. These are not reasons to avoid likelihood, but they are reasons to check convergence, inspect curvature, and report uncertainty rather than treating a single number as certain.