Digital and Software Environments › Verifiable Actions and Environment Feedback

Many software effects can be checked through an additional observation, but a check is evidence only for the variables and conditions it covers. A useful read-back must come from an authoritative source, be authorized, have suitable consistency and freshness, and identify the result attributable to this action. Reading an optimistic client cache or a lagging replica may leave the original ambiguity unresolved.

After creating an issue, a practical verifier might read an authoritative record by the returned unique ID and check its fields, ownership, and expected version. A title appearing in a list is weaker evidence because titles need not be unique. An aggregate count can increase for unrelated reasons. In the controlled toy below, the fresh store, single writer, synchronous action, and complete local read make a count increment attributable; those assumptions must not be transferred unexamined to a live application.

Reward and verification serve different roles, but their information content depends on their definitions. A reward is a scalar used by an objective; a Boolean postcondition records whether a specified property holds. If

then $r_t$ and the Boolean $P$ encode exactly the same single bit. Neither distinguishes "missing record" from "wrong title" when both make $P$ false.

Richer diagnostics require a separate structured observation, for example

Here the components describe what the verifier established; an unavailable check should not be reported as false. A scalar reward can also encode different failures or be shaped by the designer. The useful distinction is the objective's use of a signal versus the verifier's evidence about specific properties, not a universal rule that every verifier contains more information than every reward.

Verification can supply local evidence soon after an action, while a task reward may arrive much later. But asynchronous effects can require delayed verification, and rewards can also be immediate. Tie both signals to an explicit timing and attribution convention.

Read-back can reveal failures hidden by an immediate optimistic display, when the observation channel covers them.

• Silent rejection: the server rejects the request but the UI does not show the error. For example, the application might reject an expired session, an unsupported schema version, or an action disabled by a feature flag. The click alone does not establish its effect.
• Optimistic UI divergence: the client renders a success state before the server responds. The server may still fail. Verification must read the server state, not the client's optimistic rendering.
• Idempotency violations: retrying an action without an idempotency key can duplicate a purchase, a message, or a database row. Verification can detect the duplicate.
• Permission masks: an action appears available, but the server returns 403, indicating refusal. Check the relevant intended effect rather than assuming the entire system is unchanged; logging, concurrent actions, or earlier partial effects may still alter state.
• Delayed effects: the effect is real, but it has not yet occurred. Verification at step $t+1$ sees nothing, while verification at step $t+10$, with $t+10 < T$ where $T$ is the rollout horizon, sees the result. Timing matters when the effect is real but delayed.

The listed checks are conditional. A stale response may conceal a duplicate, a permission-limited list may omit an existing record, and an absent queued effect may mean "not yet" rather than "failed". A terminal status, version, idempotency identifier, or bounded polling convention may be needed. Verification does not grant access to otherwise unauthorized state.

Verification is not free. Local assertions and membership tests consume local resources but need not make a remote request; request-based checks can face latency, rate limits, or noisy responses. An agent that verifies after every action can spend more effort checking than acting, especially when each check requires reading a large list of records. Scanning thousands of rows to see whether one was added can be costly in a remote or metered collection; the cost depends on data size, pagination, consistency requirements, and the available budget.

Choose checks according to risk and information quality. High-impact actions require appropriate precondition and authorization checks even when a model is confident; low entropy is not permission to skip them. Read cost, rate limits, freshness, and the availability of a safe recovery all matter.

One possible heuristic triggers an extra observation when

Here $h_t$ is the available history and $\tau$ uses the same entropy units as $H$: bits for logarithms base two, nats for natural logarithms. The strict inequality means equality alone does not trigger this rule. Predictive entropy can mix aleatoric and epistemic uncertainty, and a poorly calibrated model can be confidently wrong.

This is a heuristic, not a value-of-information calculation. A decision-theoretic alternative estimates how the proposed observation changes expected downstream utility, then compares that benefit with its cost and risks. An observation with little entropy reduction can still be crucial for an expensive decision; a high-entropy variable can be irrelevant. This connects to uncertainty in Part IV and the earlier active-perception and dual-control discussion in Part VII.

Task success is one useful evaluation metric for software agents: did the agent complete the goal? This is useful but reductive. A single task-success number folds together at least four distinct capabilities:

1. The quality of the transition model, if one is used.
2. The quality of the planning or policy on top of it.
3. The ability to observe relevant state, including hidden parts.
4. The ability to recover from failures.

Two agents can both achieve 60% task success for entirely different reasons. One has a great model and a bad recovery policy; the other has a bad model but verifies and corrects on every step. The first is a good model applied imperfectly, and the second is a weak model compensated by a strong loop. The same score tells us nothing about which of these is happening, and so it tells us nothing about where to invest effort next. To understand why an agent succeeds or fails, we have to measure the components separately. We will develop this idea systematically in Part XI: Evaluation and Understanding, especially in the chapter on the world-model evaluation ladder.

A related point is that task success mixes predictive fidelity, representation quality, uncertainty quality, and decision usefulness into one number. A model can predict well and represent badly, or represent well and predict no better than chance on the specific transitions the task requires. Disentangling these is a prerequisite for making sense of benchmark results. Instrumented software fixtures can support separate evaluation when they provide explicit reference measurements for those components; representation quality is not automatically observable in every software task.