Scientific, Medical, and Social Worlds › A Worked Example: Two Models, One Intervention

We can make the central distinction concrete with a small simulation. Its numerical computation runs quickly once the environment is loaded; cold startup and figure rendering are separate costs. The experiment has four parts:

1. Simulate a compartmental epidemic from a known mechanistic model, and record only the weekly case counts.
2. Fit two very different models to those counts: a mechanistic susceptible-infectious-recovered model, and a flexible phenomenological autoregression on log cases.
3. Introduce an intervention that was absent from the observed history, and see what each model predicts.
4. Vary the recovery rate while refitting transmission and initial prevalence, and compare nearly unchanged training errors with different post-intervention totals.

Everything is deterministic given a fixed random seed, and everything is synthetic. The numbers below carry no medical or policy meaning. They illustrate an inference problem, not an epidemic.

The compartment model includes an action-responsive transition, but that does not make its fitted parameters correct. We will examine a shallow profile of the fitting loss over recovery rates. The autoregression, as implemented here, has no contact-action input. A different learned model could include and learn such an input; this comparison does not prove that learned transition models require hand-written mechanisms.

We observe 25 weekly reports indexed 0 through 24. All forecasts are prospective: the first unseen week is 25, when a 70 percent contact reduction begins, and the 20-week forecast window ends at week 44. The reduced-contact trajectory is withheld from fitting. The action timing and every model's plotted week must agree.

The simulator is a discrete-time SIR model. The state $s_t=(S_t,I_t,R_t)$ contains susceptible, infectious and recovered counts at the start of week $t$. The action $a_t=m_t$ multiplies the transmission rate $\beta$. Weekly incidence is $\beta m_t S_t I_t/N$, so the noisy observation model is action-conditioned, $p(o_t\mid s_t,a_t)$. All three compartments are latent to the fitted models; the simulator retains them internally. The recovered count $R_t$ is distinct from the effective reproduction number, which for $\gamma>0$ we denote $\mathcal{R}_{\mathrm{eff},t}=\beta m_t S_t/(\gamma N)$. Zero recovery is still a valid simulation setting, but this finite ratio is then undefined.

The compartment flows use weekly coefficients in [0,1] and a contact multiplier in [0,1]. For valid nonnegative compartments summing to $N$, the infection flow cannot exceed $S$ and the recovery flow cannot exceed $I$. Updating $R$ explicitly closes the population budget. In floating-point arithmetic, the implementation checks raw flows and states against the stated tolerance before bounding tiny endpoint errors; it rejects substantive violations rather than masking them. If a state crosses zero or $N$ only by roundoff, the largest compartment is reconciled with the two bounded others. This is a sufficient safe scope for our selected discrete model, not a general-purpose continuous-time epidemic integrator. The default return is a length-steps incidence array; optional state output has shape (steps+1,3).

We generate an unmodified trajectory for observations and a second trajectory with contact reduction at week 25. Before that week, both coincide. Reports multiply incidence by independent lognormal noise with log standard deviation 0.25. The conditional median of a report equals incidence; its conditional mean is larger by the factor $\exp(0.25^2/2)$. This illustrative observation process is not a surveillance calibration. Forecast totals below compare latent incidence, not noisy future reports.

These reports mostly rise over the observed window, with noise around the underlying trajectory. Log-scale residuals match our selected multiplicative observation process; they are not the uniquely correct loss for every count-forecasting task. We will examine how well this limited record constrains recovery rather than infer identifiability from the apparent trend.

The mechanistic fit minimizes squared log residuals for the selected observation model. Its three free parameters are the transmission rate, the recovery rate, and the initial prevalence. All three have meanings in the simulator: transmission, recovery and initial infectious prevalence. Initial incidence depends jointly on transmission and initial prevalence, so one incidence measurement does not separately identify both.

The competing AR(1) model regresses the next log report on the previous log report. Its two fitted mean coefficients are an intercept and slope. The code does not estimate or use an additive noise-variance parameter. One-step fitted values use observed previous reports; prospective forecasts recurse from the last observed report at week 24, with their first output at week 25. The model has no contact-action input. We also define an explicitly ad hoc output-scaled variant; it is not learned or causally identified.

The SIR forecast applies the contact reduction to the transmission flow. The unchanged AR forecast lacks that input, while the ad hoc variant multiplies each forecast output by 0.30 without changing its recursion. All three predictions now cover exactly the same unseen weeks. Giving the scaled AR an output adjustment does not validate its transition response.

The reported SIR training log RMSE is about 0.184, below the selected log-noise standard deviation of 0.25; the AR(1) value of about 0.250 is about that noise scale. They use different residual protocols: SIR is fitted recursively over the whole window, whereas AR(1) uses observed inputs for one-step predictions. This is not an apples-to-apples held-out ranking. The fitted recovery rate lands at its lower bound, and the resulting reproduction ratio is far from the simulator's true ratio. An action-responsive structure can therefore still yield a poor intervention forecast.

The first figure shows the fit window on a log scale. Both fits follow the reports, but a visual comparison on one history is neither a likelihood-equivalence proof nor an intervention test.

Now we apply the intervention. The mechanistic model reduces its transmission parameter, and the autoregression has nothing to reduce, so it continues on its learned trend. The scaled variant gets the intervention handed to it directly, as a multiplicative reduction in predicted cases, without any of the dynamical knock-on effects. The fitted incidence initially rises before a shallow decline; the executed output reports its peak week. In this simulator, the threshold for growth of infectious prevalence is $\mathcal{R}_{\mathrm{eff},t}=\hat{\beta}m_t S_t/(\hat{\gamma}N)$, including susceptible depletion. It is not just $\hat{\beta}m_t/\hat{\gamma}$, and an instantaneous prevalence threshold does not by itself determine the slope of incidence. We therefore inspect the actual trajectory rather than infer its shape from a ratio.

Three things are worth reading off this figure carefully:

• The fitted SIR incidence starts in the same broad range as the truth, but initially rises while the truth declines. Its net decline over the window is much shallower. The estimated parameters differ substantially from the simulator's true parameters; action responsiveness alone has not established accurate intervention dynamics.
• The unscaled autoregression diverges almost immediately and continues to diverge. It has been asked a question it has no machinery to answer, and its answer is implicitly "nothing changes."
• The ad hoc scaled variant is the most instructive curve of the three. It was given the intervention explicitly, with the same multiplicative factor the mechanistic model used. It still fails, for a deeper reason: in a mechanistic model, reducing transmission does not merely reduce this week's cases; it also prevents those infections from generating further infections next week and depletes the susceptible pool more slowly. The effect compounds. In this simulator the intervention changes the trajectory over time, so a constant output factor does not reproduce its response. That is not a general claim about scaling: proportional trajectories could be related by a constant factor. This result concerns the specified AR baseline, not all learned or statistical simulators.

We can now probe parameter uncertainty directly. At each fixed recovery rate $\gamma$, refit transmission $\beta$ and initial prevalence $I_0$ to the same observations, then record training log RMSE and post-intervention totals. During approximately exponential early growth, the net growth rate is primarily related to $\beta-\gamma$, not the ratio $\beta/\gamma$. Here the fitted loss changes little across several recovery rates even though the ratio and future totals change. This is a shallow profile of the loss under noisy, limited observations, not an exact non-identifiability theorem. Every tested refitted trajectory has a net decline in this post-intervention window.

Every row of that table describes a model with a similar training error. Their reproduction ratios and intervention totals differ. The executed output below computes the ratio of the largest and smallest total rather than hardcoding it in the prose. This demonstrates a shallow fitting-loss profile relevant to the forecast, not an exactly flat likelihood ridge.

The next two figures share the fixed recovery-rate axis. One shows the four training errors, and the other shows their post-intervention totals. Their separation is visible on a linear total-incidence axis.

The last figure isolates parameter uncertainty within one transition structure. All four settings decline over this window, but they disagree about how much incidence remains. Independent information about recovery or transmission could constrain this shallow fitting-loss profile. Longer or more informative observations can also help; there is no claim that additional observational data can never identify these parameters.

Three caveats on the experiment, stated plainly:

• This is a synthetic system with three state variables and a known generative process. Real outbreaks can involve age structure, contact networks, asymptomatic transmission, behavioral feedback, waning immunity, and reporting delays. None is modeled here.
• The AR(1) baseline is deliberately weak. A stronger learned model could include action inputs and suitable training data; this example does not evaluate such a model.
• Nothing here is a statement about any real disease or any real policy. The exercise demonstrates forecast ambiguity within its selected models and synthetic data. Drawing a public health conclusion from it would misuse the example.