Scientific, Medical, and Social Worlds › Domain Constraints, Uncertainty, and Misuse Risk

The domains above reuse several engineering practices. None is exclusive to scientific world models, and none establishes validity alone. They organize checks on a model's represented mechanisms and the query it is asked to answer.

Mechanistic constraints include unit consistency, nonnegative populations or concentrations, applicable symmetries, and balance laws with their sources, sinks, and boundary fluxes. A soft penalty discourages a violation; a hard parameterization can prohibit a specified violation within the represented model. Either can be appropriate depending on scope and numerical implementation. A hard energy constraint does not guarantee correct forces, trajectory accuracy, or freedom from every form of drift. For an open or driven system, forcing a constant-energy rule can itself be wrong. Test the discretized implementation rather than infer its properties from the continuous equations alone.

A surrogate approximates an expensive calculation to make repeated evaluation affordable. Its accuracy is defined relative to its reference, which can itself be approximate. A surrogate trained over a narrow regime needs separate evidence for a new composition, forcing, or patient group. Some surrogates include uncertainty models or rejection rules; smoothness and fast evaluation neither guarantee confidence nor imply an inability to signal failure. Compare interpolation, extrapolation, and decision-relevant error explicitly.

A hybrid keeps a selected mechanistic component and learns another, such as an unresolved tendency or residual correction. This makes assumptions inspectable but does not partition the system into a guaranteed-correct core and a harmless learned part. Components interact during rollout. Errors in a learned closure, a mechanistic approximation, or numerical integration can affect the whole trajectory. Validate the combined system and its supported interventions.

For a stated model and task, it helps to distinguish several sources:

• Observation or process uncertainty describes measurement variation or modeled randomness, for example in $p(o_t\mid s_t,a_t)$. Better sensors or richer state can change what is treated as irreducible.
• Parameter uncertainty concerns quantities within a selected family. A posterior depends on its likelihood and prior. Bootstrap and ensemble estimates depend on the resampling, variation, data, and fitting assumptions actually used; neither label implies a Bayesian posterior.
• Model-form uncertainty concerns alternative mechanisms or approximations. Candidate-model comparisons can represent included alternatives but need not cover omitted ones.
• Numerical uncertainty concerns finite resolution, time stepping, and floating-point calculations. It can matter even when the mechanistic family is fixed.

These categories have no universal ranking by size or difficulty. Their importance can change with horizon, data, and target. Report which variations an interval represents and which it omits. A narrow interval can be appropriate for a narrow conditional query, yet inadequate if presented as covering every source of deployment error.

An ensemble can vary initialization, data, parameters, or model form. Its spread describes the chosen variation; agreement among models with shared limitations is not evidence of total uncertainty. Conformal prediction supplies a different, coverage-oriented guarantee. Under an appropriate exchangeability assumption and correctly constructed calibration procedure, a prediction set can satisfy:

$$P(Y_{\text{new}} \in \hat{C}(X_{\text{new}})) \geq 1 - \alpha$$

where:

• $Y_{\text{new}}$: the outcome for a new, unseen observation
• $X_{\text{new}}$: the covariates for the new observation
• $\hat{C}(X_{\text{new}})$: the set obtained by applying the calibrated procedure to the new covariates
• $\alpha$: the desired marginal miscoverage bound.

The probability above averages over the calibration sample and a new exchangeable example. It is a marginal guarantee, not a guarantee for every covariate value or subgroup. Time-series dependence, policy intervention, and regime shift need conditions appropriate to those settings; ordinary exchangeable calibration does not automatically apply. See Angelopoulos and Bates. A conformal set can retain marginal coverage around a misspecified predictor by becoming wide. That does not identify an intervention response.

For both ensembles and calibrated sets, uncertainty statements should name the query and distribution. An ensemble using one architecture may miss an omitted mechanism. A candidate-form ensemble can vary some mechanisms but still leave others out. No method's label establishes that every source has been covered.

Choose tests that match intended deployment. A held-out random split can assess prediction under a similar distribution; it is not sufficient for a materially new regime. Hold out relevant temperatures, pressures, regions, periods, subpopulations, or forcing levels when those are the intended shifts. Also test actual intervention response where evidence is available. An ordinary regime holdout can expose a failure, but passing one does not establish every unobserved action or boundary condition. Useful implementation diagnostics include:

• Balance checks. With the modeled input, output, and boundary fluxes accounted for, does the discrete mass or energy budget close to its stated tolerance?
• Symmetry checks. Does the prediction transform as required when coordinates or valid equivalent labels change?
• Gradient checks. If forces are defined by an energy gradient, do analytic and finite-difference gradients agree within numerical tolerance?
• Domain checks. Do populations remain nonnegative, and do concentrations and action inputs stay within the model's declared domain?
• Response checks. Where a monotonicity or dose-response restriction is actually justified, does the implementation respect it under the specified conditions?

Some internal checks do not require a measured reference trajectory. The PK transfer balance follows from its own equations; an implementation violating it is inconsistent even before empirical validation. These tests still require correct scope: energy can change in a forced system, populations can cross a boundary, and real dose responses need not always be monotone. Internal consistency is necessary for the stated model, not sufficient evidence that the model represents the world.

Observational validation establishes performance for the tested observational query and evaluation conditions. An intervention claim needs an identification argument or matching experimental evidence, not only residual accuracy. In a binary treatment example, conditional exchangeability is one condition:

$$\{Y(1), Y(0)\} \perp A \mid \mathbf{X}$$

Consistency, well-defined strategies, positivity on the target support, and any relevant interference conditions are also needed for the standard identification route. When these conditions hold, observational data can contribute to estimating an interventional effect. Without them, an accurate prediction score does not settle that effect. Distinguish estimation uncertainty under stated assumptions from uncertainty about whether the assumptions hold.

Several misuse risks follow from losing those distinctions. Precision theater presents a narrow interval without its conditional scope or omitted uncertainties. Policy laundering treats a model as independent authority for a choice whose values and objectives remain unstated. Medical overreach substitutes a forecast for an identified and validated treatment claim. Social-simulation overreach treats a synthetic response distribution as a measured population. Dual-use concerns arise when a generation system can propose harmful as well as beneficial candidates. Feedback risk arises when predictions or actions change later data. These are reasons to document scope, oversight, and refusal conditions, not predictions that every deployment will fail in these ways.

The practical discipline is to state assumptions, test the intended regimes and actions where feasible, name the uncertainty sources represented, and keep predictions separate from choices under a loss. When evidence does not identify the query, a range, sensitivity analysis, or explicit nonidentification statement can be more informative than a single fitted trajectory.