Sim-to-Real Deployment and Continual Correction › Safe Data Collection and Update Policies

Adaptation needs information about the deployment plant, and collecting that information can have physical consequences. Desired coverage may include conditions that are hazardous to visit. Observations collected within a safe region can sometimes identify a broader dynamics model under justified structural assumptions, but that inference still needs validation. Plan collection around the information needed and the permitted operating region, not coverage at any cost.

Safe data collection is not blind online exploration. The phrase "let the agent explore and improve" describes a research protocol, not a deployment policy. Safe data collection in practice means:

• A stated operating envelope: position, velocity, force, current, and temperature limits that define where the system is allowed to go, justified independently of the learned model being updated. Relevant errors in a model-derived envelope can invalidate its safety argument, although not every model error changes or defeats a conservative bound. Keep the independent physical limits and the assumptions behind any model-based bound explicit.
• Independent safeguards: hard limits enforced outside the learned component. If the learned component can disable its own safety monitor, it is not a safeguard. Independence means the safeguard's authority does not flow through the thing it is guarding.
• Supervised collection: a human able to stop the run, with a stated stop criterion. A supervisor without a criterion is a bystander; the criterion is what makes supervision actionable.
• Bounded excitation: an excitation signal designed to cover the modes you care about, bounded to stay inside the envelope, rather than a random walk that might find a corner of the state space you did not consider.
• Purposeful coverage: without justified prior information about actuator saturation, adaptation data must activate that regime to identify its limit. Collect such data only within approved bounds and under supervision. If a known saturation law is supplied instead, state and validate that prior structure; unactivated data alone do not establish it.

The guarantees here are conditional. Berkenkamp et al. (2017) developed high-probability stability verification and safe exploration for deterministic discrete-time dynamics. Their requirements include Lipschitz dynamics and policies, an initially safe policy/set, a Lyapunov argument and calibrated dynamics confidence bounds. Their GP exploration analysis additionally assumes bounded RKHS model error, fully observed state and suitable measurement noise. The paper demonstrates a simulated inverted pendulum, not universal hardware safety. This proxy establishes none of those premises. A theorem can be valid conditionally while its guarantee remains unestablished for a particular deployment.

These are different operations with different risks and different evidence requirements. Conflating them is a common source of confusion in deployment reviews, where "we added adaptation" can mean any of the four.

• Continual model correction: updating the predictor while leaving controller code fixed. A predictor change can still change commands substantially; assess the resulting closed loop, not just its prediction metric.
• Policy adaptation: updating the action-selection rule. Its risks depend on the update and safeguards; there is no universal risk ranking relative to predictor updates.
• Parameter identification in this chapter: re-estimating the oscillator's selected interpretable parameters, one restricted form of system identification. System identification can also involve choosing a model structure or fitting a larger, less interpretable dynamics model. A small parameter vector aids inspection but does not itself establish identifiability, a safe update or negligible control impact.
• Monitoring: detecting a departure without directly updating a model. Responses to an alarm can still affect control. Monitoring need not be the only trigger for data collection or scheduled evaluation.

The proxy fits parameters and a residual from the same adaptation episode before its monitoring demonstration. Residual fitting uses the nominal predictor, not the identified parameter vector; detection is not a programmatic precondition for either fit. A deployment workflow can gate updates on monitoring or scheduled evaluation, but that orchestration is not implemented here.

An updated model is a versioned artifact, and the deployment should be able to say which version produced which behavior. Concretely:

• Identity: give each promoted artifact a version identifier and bind it to recorded code, data and configuration identities. A content hash identifies bytes; a tag requires a controlled binding to the intended artifact. Neither supplies missing training records by itself. The serving configuration selects an artifact by identity, while an audit uses the associated records to reconstruct the promotion decision.
• Gating record: the held-out evaluation that justified promotion, stored with the model.
• Rollback: retain a previous validated artifact together with its compatible configuration and dependencies. Test the switchback protocol, including session-state compatibility and fallback behavior. Changing a pointer can select the prior artifact, but does not by itself establish a safe or working rollback.
• Provenance: adaptation data is stored with its collection conditions, so a future failure can be traced to the conditions the model was fitted to.
• Auditability: someone other than the person who promoted the model can reconstruct why.

Two failure modes need separate checks. Forgetting or retained-condition regression occurs when an update degrades performance on previously covered conditions; a severe loss of previously learned capabilities is called catastrophic forgetting. Replay, regularization and retained-condition evaluation can help, but do not guarantee its absence. Undetected drift is a change that the monitor fails to flag, whether gradual or abrupt. Scheduled validation supplements event-triggered checks; its ability to detect drift still depends on the tested conditions and metrics. Neither failure is necessarily sudden or visible without evaluation.

Security, Ethics, Privacy, and Governance uses provenance and accountability records to investigate scoped questions about training and deployment. Missing records can limit attribution or confidence without making every review impossible; other relevant evidence may remain available. Training and Inference Systems distinguishes reproducible training continuation from serving lifecycle and cache identity. A retained, compatible model artifact can support rollback even when its full training manifest is incomplete. Conversely, a complete manifest cannot restore an artifact that was not retained or cannot run in the current deployment. Record promotion evidence and test recovery separately: they answer different questions.