Security, Ethics, Privacy, and Governance › Code Implementation

The synthetic experiments below are mechanism probes, not reproductions of attacks against a real system. Computation, displayed outputs, and presentation-only plots are separate, and random generators are seeded. The plant state is directly observed here: $o_t=s_t$, so no belief-state estimator is implemented. A predictor $\hat f_\theta(s_t,a_t)$ estimates the next state; closed-loop tests use horizon $H$. This fully observed scalar plant is not a video generator or evidence about deployed robot privacy. A probe may fail to demonstrate the anticipated effect. We retain such negative results rather than adjusting seeds or claiming leakage that the outputs do not show.

We begin with one environment. We will use a one-dimensional integrator with a bounded control input: a crude model of a velocity-controlled joint or a robot that moves at a commanded speed. This small plant lets us study how an incorrect control gain changes closed-loop behavior. A linear plant is chosen deliberately: not because real world models are linear, but because parameter corruption and local closed-loop instability can already occur in a linear system. Closed-form analysis lets us derive attack conditions under explicit feature, label and row-budget constraints. The demonstrations below do not solve every possible attack against the learner.

The behavior policy adds excitation to state feedback and then clips the executed action: $a_t = \operatorname{clip}(-K s_t + \eta_t, -2, 2)$. Here:

• $s_t$: the state at time $t$
• $a_t$: the action taken at time $t$
• $K = 1.0$: the feedback gain, which sets how strongly the action responds to the current state
• $\eta_t \sim \mathcal{N}(0, 0.8^2)$: a fresh Gaussian excitation signal drawn independently at each step, whose variance $0.8^2 = 0.64$ sets how strongly the recorded actions diverge from a pure function of the state.

The feedback term makes the recorded actions correlated with the states, while the independent excitation lets actions vary at a given state. Clipping changes the executed-action distribution; the actions themselves are not Gaussian. As we discussed in Part II, Ch 3: System Identification, identifying both coefficients requires enough independent information in the state-action design. The realized two-column design has full rank in this seeded example. Adding noise alone does not guarantee that condition for every finite dataset or clipping regime.

The poisoning experiment below shows why this support also matters to security. The same design matrix enters estimation and the attack equations. How much protection it provides depends on the clean data scale and the adversary's permitted feature values, labels and row budget, not on identifiability alone.

Fitting a linear transition model is a ridge regression, which has a closed form and therefore gives us an exact window into what an attacker can and cannot do. A closed form matters here because it means we can write down, in advance, how the fitted parameters will change in response to any perturbation of the data. That is precisely the quantity an attacker optimizes, and having it analytically lets us relate fitted-parameter changes to the trajectories we simulate below.

The estimated coefficients are close to their true values in this seeded sample. The held-out one-step RMSE is near the process-noise standard deviation. These are sample estimates, not confidence bounds on the coefficients or a proof of identifiability for all collection policies.

Now we use the model in a controller to examine the difference between predicting well and deciding well. We give the agent a setpoint and let it solve the one-step planning problem in closed form. The controller inverts the learned dynamics. The model predicts the next state from the current state and the chosen action:

where:

• $\hat{s}_{t+1}$: the model's predicted next state
• $s_t$: the current state at time $t$
• $a_t$: the chosen action at time $t$
• $\hat{a}_{\text{state}}$: the learned state coefficient
• $\hat{b}_{\text{gain}}$: the learned action gain

To drive that predicted next state to the target $s^\star$, the controller inverts this relationship and chooses

where:

• $s^\star$: the setpoint, the state the controller wants to reach
• $s_t$: the current state at time $t$
• $\hat{a}_{\text{state}}$: the learned state coefficient of the dynamics model
• $\hat{b}_{\text{gain}}$: the learned action gain of the dynamics model
• The division by $\hat{b}_{\text{gain}}$ inverts the learned action effect; a gain error can alter commands, depending on the state, numerator, and clipping

The structure of this controller explains the poisoning target. The numerator cares about where the state is and where we want it to be. The denominator represents how much a unit of action moves the state. An incorrect $\hat{b}_{\text{gain}}$ can change a nonzero unsaturated command, but it need not change a zero command or one already clipped to the same bound. Nor does every gain error compound: with true gain $B=0.5$, learned state coefficient one and learned gain one, the ideal noiseless error halves at each step. Amplification or instability depends on the resulting closed-loop pole, which we derive below.

With this fitted model, the first command is near the control limit but does not quite saturate. The state then stays near the setpoint under the specified disturbance sequence. This finite run is the reference behavior, not a claim that every disturbance or initial condition is safe.