Prediction Intervals and Split Conformal Prediction › Advantages

The coverage bound holds in finite samples under exchangeability of the held-out scores. It does not require a Gaussian error distribution or a correctly specified predictor. A fixed Gaussian margin can under-cover when its assumptions fail. Split conformal instead calibrates the chosen score, including an infinite threshold when the calibration sample is too small for a finite cutoff. The resulting sets can be wide or uninformative while remaining marginally valid.

Conformal prediction wraps a fitted model. Linear regression, a boosted ensemble, or another predictor can supply the score, provided fitting and selection are settled using fitting data only. The residual-score version needs predictions on $n$ calibration examples and a sort of $n$ scores, conventionally $O(n\log n)$ time for sorting. This extra work depends on the predictor and calibration size; it is not universally negligible.

Changing the score can change the shape and adaptivity of the output. Absolute residuals give constant-width regression intervals; positive input-dependent scaling can redistribute width; class-probability scores give label sets. All use the same marginal rank argument when their scores are fixed before calibration. Better probability calibration or a different score may improve set efficiency, but neither guarantees smaller sets or better subgroup coverage in every problem.

Interval width and set size can inform operational rules, such as routing broad intervals to human review. Such a rule is an additional selection step: marginal coverage of the original procedure does not imply the same coverage among automatically accepted cases. Evaluate the selected cases separately and distinguish a useful triage signal from a calibrated conditional error rate.