Perceptual and Generative Evaluation › Summary

Perceptual and generative evaluation answers a specific question: how closely do a model's predicted observations resemble the observations the world produces, individually and in distribution. The chapter developed this question in four layers.

• Reconstruction and predictive accuracy. MSE, PSNR, and SSIM are paired full-reference metrics with different score directions. Expected squared error is minimized by the conditional mean under finite-second-moment assumptions; this theorem does not establish an optimum for expected PSNR or SSIM. In our separated two-mode example the mean looks faded, while a sharp prediction matching the realized mode wins on that draw. Declare dynamic range, averaging conventions, and SSIM preprocessing.
• Perceptual and distributional metrics. LPIPS compares images in the feature space of a pretrained network with weights fit to human judgments, and inherits the backbone, preprocessing, and calibration-set dependencies of that construction. FID is the squared 2-Wasserstein distance between Gaussian fits of Inception features of real and generated distributions; FVD generalizes to spatiotemporal features. Both compare moments, not full distributions, and both depend on encoder choice, sample counts, preprocessing, and spatial resolution. FVD also depends on clip duration and frame rate; video-frame FID needs a declared frame-sampling protocol, not an intrinsic clip-length input.
• Diversity, sharpness, and multimodality. Distributional checks complement point losses for stochastic futures. Nested best-of-$K$ errors cannot increase as $K$ grows, but this oracle rule does not test calibration or deployable selection. High variance is not coverage. Precision/recall depends on its feature-space estimator. Pooling can hide balanced action reversals; stratification exposes our separated-mean example but is not a complete conditional test. Conventional plug-in feature distances can have model-dependent finite-sample bias; sample size, estimator, and uncertainty all matter.
• Human judgment and learned evaluators. Direct ratings require a stated question, presentation and sampling protocol, and uncertainty appropriate to trajectory and rater dependence. Learned evaluators need held-out agreement and domain validation. Their scores remain proxies, particularly when the same judge is used to train or select a generator.

Perceptual and generative scores provide evidence about specified observation properties. They do not certify state, physics, causality, memory, or decision quality. Use them alongside tests designed for those properties. The next chapter examines what the model preserves and predicts beyond visual similarity.