Digital and Software Environments › A Worked Experiment: Optimistic vs Verified Predictors
World Models HandbookAll chapters Digital and Software Environments › A Worked Experiment: Optimistic vs Verified PredictorsOne section · Sections in this chapter6 / 9 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let us make this concrete. We will build a deterministic local issue tracker with two hidden facts, a session validity flag and a team membership roster, and then compare three agents:
- Optimistic: predicts transitions using only the visible form fields.
- Verified: uses the same model but performs a read-after-write correction after every action.
- Verified + recovery: additionally attempts a targeted recovery strategy when a save fails.
The three agents share an identical transition model, which isolates the contribution of verification and recovery from the contribution of the model itself. This is deliberate: the point of the experiment is not to demonstrate that a small state machine is inadequate, but to show that the same inadequate model can succeed or fail depending on the loop around it.
For each run we report these quantities separately:
- Raw one-step prediction accuracy: exact agreement of the prediction made before the action with the resulting observation, scored before any correction.
- Post-correction belief agreement: agreement after the optional read-back correction.
- Final observable drift: the number of mismatched components of the final observation triple; this is not full hidden-state drift.
- Task completion: whether the intended title-assignee tuple exists in the fresh, single-writer store.
- Rejected Save attempts: every actual Save click that appends no row, including retries.
- Successful recoveries and recovery-action cost: failed primary saves repaired by the retry policy, and additional actions it executes.
The environment exposes the observation
where is the issue count. Its full toy state includes the complete issue list, form fields, session-validity flag, and the roster set . A Boolean membership answer for the current assignee is not enough to predict all future assignee choices; the roster set is the relevant fixed environment parameter.
The local fixture has no queue, clock, concurrent writer, or network. Those omissions make its transitions deterministic. They are deliberate experimental boundaries, not claims about real trackers.
Let us define the environment and the optimistic model.
import numpy as np # noqa: E402
TEAM = {"ada", "bob"} # hidden server-side roster
class IssueTracker:
"""A tiny local issue tracker with hidden session and team state."""
def __init__(self, session_valid: bool):
self.session_valid = session_valid # hidden
self.title = None # visible form field
self.assignee = None # visible form field
self.issues = [] # persistent store
def observe(self):
return (self.title, self.assignee, len(self.issues))
def step(self, action):
kind, arg = action
if kind == "type_title":
self.title = arg
elif kind == "type_assignee":
self.assignee = arg
elif kind == "relogin":
self.session_valid = True
elif kind == "click_save":
ok = (
self.session_valid
and self.title
and self.assignee
and self.assignee in TEAM
)
if ok:
self.issues.append((self.title, self.assignee))
return self.observe()The class exposes only the visible triple through observe(). The click_save branch is where things get interesting: it is silently a no-op unless every hidden condition holds. In state-transition terms, the transition conditioned on appends to the issue list only when and ; otherwise the transition is the identity on the persistent store. This is a deterministic kernel, so the distribution places all its mass on a single next state.
The optimistic model is a transition function over the visible triple. Formally, it approximates
using only , ignoring the hidden components of . It reasons about required fields but has no access to session validity or team membership.
def optimistic_step(state, action):
"""Transition model over visible state only."""
title, assignee, n_issues = state
kind, arg = action
if kind == "type_title":
title = arg
elif kind == "type_assignee":
assignee = arg
elif kind == "click_save":
if title and assignee:
n_issues += 1
return (title, assignee, n_issues)This is a hand-written optimistic predictor, not a learned model. It models the required visible fields but ignores session validity and the roster. Its Save prediction is wrong in the two hidden-precondition failures. The comparison therefore illustrates feedback around an imperfect predictor; it does not estimate the performance of a trained GUI or API model.
The driver executes the same three primary actions in each mode against a fresh tracker. It logs a raw prediction before every real environment action, then stores the resulting observation and optionally corrected belief. Recovery adds relogin and one Save retry, each logged and scored separately. The policy tries a bounded repair without first identifying the cause; it can therefore repeat a failure when the assignee is invalid.
# NOTE: this driver mutates environment state and runs the simulation loop, so it is
# explicitly not theme-rerunnable; the plot cells below only read results from `results`.
def run(session_valid, actions, mode):
if mode not in ("optimistic", "verified", "verified_recovery"):
raise ValueError("Unknown policy mode")
if not actions:
raise ValueError("This experiment requires a nonempty action sequence")
verify = mode in ("verified", "verified_recovery")
recover = mode == "verified_recovery"
env = IssueTracker(session_valid)
belief = env.observe()
steps = []
recovered = 0
def execute(action, phase):
nonlocal belief
before = len(env.issues)
raw_prediction = optimistic_step(belief, action)
observation = env.step(action)
# Score the prediction before correction; keep both in the audit log.
raw_match = int(raw_prediction == observation)
belief = observation if verify else raw_prediction
record = dict(
action=action,
phase=phase,
raw_prediction=raw_prediction,
observation=observation,
belief=belief,
raw_match=raw_match,
belief_match=int(belief == observation),
rejected_save=int(
action[0] == "click_save" and len(env.issues) == before
),
)
steps.append(record)
return record
for action in actions:
record = execute(action, "primary")
if recover and record["rejected_save"]:
execute(("relogin", None), "recovery")
retry = execute(action, "recovery")
recovered += int(not retry["rejected_save"])
final_observation = env.observe()
drift = sum(int(a != b) for a, b in zip(belief, final_observation))
primary = [step for step in steps if step["phase"] == "primary"]
# The goal is derived from the requested form values, not a hardcoded result.
goal_title, goal_assignee = None, None
for kind, arg in actions:
if kind == "type_title":
goal_title = arg
elif kind == "type_assignee":
goal_assignee = arg
return dict(
steps=steps,
primary_raw_accuracy=sum(s["raw_match"] for s in primary)
/ len(primary),
raw_accuracy=sum(s["raw_match"] for s in steps) / len(steps),
belief_agreement=sum(s["belief_match"] for s in steps) / len(steps),
action_count=len(steps),
final_drift=drift,
task_complete=int((goal_title, goal_assignee) in env.issues),
save_attempts=sum(s["action"][0] == "click_save" for s in steps),
invalid_actions=sum(s["rejected_save"] for s in steps),
recovered=recovered,
recovery_actions=sum(s["phase"] == "recovery" for s in steps),
)Three scenarios cover the interesting cases: a happy path, an expired session, and an unknown assignee who is not on the team roster.
scenarios = {
"happy": (
True,
[
("type_title", "Login fails"),
("type_assignee", "ada"),
("click_save", None),
],
),
"expired_session": (
False,
[
("type_title", "Crash on save"),
("type_assignee", "bob"),
("click_save", None),
],
),
"unknown_assignee": (
True,
[
("type_title", "Flaky test"),
("type_assignee", "carol"),
("click_save", None),
],
),
}
modes = ["optimistic", "verified", "verified_recovery"]
results = {}
for name, (sv, acts) in scenarios.items():
for mode in modes:
results[(name, mode)] = run(sv, acts, mode)The table reports primary-action accuracy over the common three actions, all-action raw accuracy with its denominator, post-correction agreement, observable drift, task completion, every rejected Save, repaired saves, and recovery-action cost. Recovery rows can contain five actions; the other rows contain three. The all-action scores therefore have different denominators and should not be mistaken for identical test sets.
scenario mode primary raw N belief drift done saves reject repair extra --------------------------------------------------------------------------------------------------------- happy optimistic 1.00 1.00 3 1.00 0 1 1 0 0 0 happy verified 1.00 1.00 3 1.00 0 1 1 0 0 0 happy verified_recovery 1.00 1.00 3 1.00 0 1 1 0 0 0 expired_session optimistic 0.67 0.67 3 0.67 1 0 1 1 0 0 expired_session verified 0.67 0.67 3 1.00 0 0 1 1 0 0 expired_session verified_recovery 0.67 0.80 5 1.00 0 1 2 1 1 2 unknown_assignee optimistic 0.67 0.67 3 0.67 1 0 1 1 0 0 unknown_assignee verified 0.67 0.67 3 1.00 0 0 1 1 0 0 unknown_assignee verified_recovery 0.67 0.60 5 1.00 0 0 2 2 0 2
Reading the row for expired_session under optimistic: the model believes the save worked, but the environment silently rejected it. Drift is 1 and task completion is 0. The model is confidently wrong. The unknown_assignee scenario under optimistic fails the same way, because the model has no idea that carol is not on the team. Recall that drift counts the number of visible components on which the model's final belief and the true final observation disagree. Writing the three components as indexed entries of the observation vector, we have:
where:
- : the final step index of the rollout
- : the -th component of the model's final predicted observation
- : the -th component of the true final observation
- : the indicator function, equal to 1 when its argument is true and 0 otherwise
- the index ranges over the three visible components (title, assignee, and number of issues)
The verified rows have post-correction agreement 1.00 and final observable drift zero because this fixture's complete local observation is copied into the belief. Their raw primary-action accuracy remains 2/3 in the failing scenarios. Verification has not made the predictor perfect; it has corrected its exposed errors. Verification alone still leaves both tasks incomplete.
For an expired session, relogin plus retry repairs the failed Save. The initial rejection remains counted, so this run has two Save attempts, one rejection, and two extra recovery actions. For an unknown assignee, both the initial Save and the retry are rejected: two Save attempts, two rejections, two extra actions, and no completed task. Recovery is useful only when its actions address the actual precondition failure.
Let us visualize the drift comparison first.
For the common primary actions, raw accuracy is in both failing scenarios for all three modes. Let be the number of real actions included in a score and the prediction recorded before correction:
Post-correction agreement uses the corrected belief instead and measures a different property. The logged records retain both values so the verifier cannot erase prediction errors from the evaluation.
The primary traces end at the first incorrect prediction. They demonstrate that an aggregate over easy typing actions can hide the decisive Save miss, not that this miss propagated over a long rollout. In recovery mode the expired-session trajectory has four correct raw predictions among five actions; the unknown-assignee trajectory has three among five. These are trajectory-dependent scores: verified modes feed corrected beliefs into later predictions, and the recovery policy also changes the actions.
Now let us examine task completion and all rejected Save attempts under the recovery policy. These counts were computed in the experiment cell above (the verified_recovery rows).
The chart retains the initial failed Save even when recovery succeeds. Relogin changes session validity but cannot add carol to the roster, so the unknown-assignee retry is rejected too. Task completion and rejected attempts answer different questions: a successful task can still incur a failed action and a recovery cost.
What the Experiment Shows
The table supports three limited conclusions.
First, the optimistic predictor's common three-action accuracy is in each failing scenario while task completion is zero. Two correct typing predictions do not compensate for the decisive Save miss. The short experiment does not demonstrate long-horizon compounding.
Second, the fixture's observation channel corrects observable drift without changing raw prediction accuracy on the common actions. This illustrates belief correction under an exact, fresh local read. It does not establish that a real authorized read covers every hidden variable or that correction increases full-task success by a particular factor.
Third, recovery and detection differ. The bounded relogin-retry policy fixes the expired session but fails on the unknown assignee, adding two actions in either case. Recovery can use search, retry, a fallback, or an explicit causal hypothesis; successful repair does not always require a named diagnosis. More targeted repair of the assignee would require authorized roster information or another justified action. A supplementary observation channel could be written
where reveals allowed assignees under the channel's permissions. This is additional evidence, not something the original observation triple already contains.