Digital and Software Environments › Web, Desktop, and Code Execution State

A world-model state $s_t$ contains the variables needed to predict the next transition, as introduced in Part II. For a software system, these may include processes, files, database records, pending events, and clock-dependent conditions. The agent usually sees only a projection $o_t$ and maintains a belief $b_t$ over relevant states from its observation-action history. An observation is evidence about state, not a replacement for state.

The state can mix discrete variables, continuous clocks, buffers, and external inputs. Observations are renderings or partial serializations of that state. Actions may change as an interface changes. Transitions can be deterministic in a controlled fixture yet uncertain to an agent that lacks the relevant state; real systems may also introduce stochastic scheduling and external events. Distinguish uncertainty about hidden state from randomness in the transition.

What typically lives in the state? We can sort the components into visible observations, hidden state, actions, asynchronous effects, and task goals.

Visible observations are what an agent can capture directly:

• Rendered pixels of the current screen.
• A DOM snapshot or accessibility tree: distinct structural and semantic projections that can include or omit different interface elements.
• HTTP API responses: status and headers, plus any structured or unstructured response payload (which may be absent).
• Console or log output from an application or a code interpreter.

Each channel exposes only part of the state. Pixels may conceal offscreen or masked values. A DOM snapshot need not describe a native dialog or all canvas content. API responses expose the endpoint's authorized projection, not necessarily the whole database. Printed output is not the interpreter's full memory. For a desktop editor, for example, the current document text may be visible while unsaved buffers, the selected file path, focus, and a pending background save affect the next action.

Hidden state is what visible observations cannot disambiguate:

• Session and authentication state: cookies, bearer tokens, expirations, refresh timers.
• Permission scopes: which rows a user is authorized to read or write.
• Feature flags and A/B assignments: which version of a page the user is currently seeing.
• Server-side persistent state: rows in a database, objects in a blob store, entries in a search index.
• Rate-limit counters, retry budgets, and quotas.
• Background jobs: scheduled tasks, asynchronous workers, webhook deliveries.

What is hidden depends on the available tools and permissions. Browser tooling may expose cookie expiry, but that alone does not establish server-side session validity or revocation status. Some interfaces reveal feature assignments or quota headers; others do not. A variable can remain unidentifiable even after many permitted observations. An agent should record what each tool can establish rather than assume a universal read-back channel.

Actions include GUI events, structured API requests, and code evaluations. A GUI event may initiate an asynchronous request; an API response may acknowledge queued work rather than completion; a code statement may spawn a process that outlives the statement. The action boundary used by a model therefore needs an explicit timing convention, such as "response received" or "job completed".

Asynchronous effects include queues, background saves, webhook deliveries, and client reconciliation. An optimistic client may later reconcile with a server response, but that reconciled display can still lag another writer or read replica. A full-state model includes pending work and time-relevant variables. A model built only from the immediate screen can miss those delayed effects.

Task goals are usually compositional. A checkout flow decomposes into "purchase item", "enter shipping information", and "pay", each with its own preconditions and postconditions. Filing a bug report decomposes into opening a form, filling in the fields, and confirming the submission. Composition matters because the semantics of an action depend on the milestone being executed: the same click is a submission in one context and a navigation in another. When we later consider hierarchical decomposition, the goal structure is what makes that decomposition natural.

The Markov property says that a sufficient full state screens off earlier history:

The practical failure is usually the approximation that a single observation is such a state. Two identical screens can correspond to different session or permission states and therefore different outcomes of the same Save click.

A history $h_t=(o_0,a_0,\ldots,a_{t-1},o_t)$ can contain clues absent from the current screen. A belief $b_t(s)=p(s_t=s\mid h_t)$ represents the remaining ambiguity. If all available histories are indistinguishable, a calibrated model can predict a mixture of outcomes; it need not pretend that one outcome is certain.

Wall-clock time, pending queues, random-generator state, and relevant external inputs belong in the state or transition convention when they affect future behavior. Including them can restore a Markov description at the full-state level. Their omission from an observation-level representation does not prove that the complete system violates the property. Changes to a UI, schema, or server implementation introduce a separate distribution-shift problem; physical environments can change too.

Code execution has several state boundaries. The interpreter contains variables, scopes, and stack frames; the surrounding execution environment contributes files, process resources, and external effects. Relevant variables can include:

• The value of every variable in every active scope.
• The current instruction and call stack.
• The contents of the file system reachable from the process.
• The working directory, environment variables, and open file descriptors.
• Any child processes spawned by the program.
• Network sockets and buffered I/O.

A debugger, instrumented interpreter, or local test harness may inspect some of this state. A printed value or exit code inspects much less. A predicate such as result == 42 gives an answer about that result under the executed conditions. A successful file-existence check provides evidence about a path at a particular time, not about its contents; a false result may also reflect an inability to inspect the path. Unit and property-based tests verify their assertions and exercised cases, not every future transition.

Controlled code execution can help separate predictive fidelity from representation quality and decision usefulness. Compare predictions against explicit observed values, test which hidden variables a representation preserves, and evaluate the decisions it supports. The comparison is only as complete as the instrumentation and assertions. This chapter executes a closed, deterministic fixture; it does not execute arbitrary agent-generated code or contact live services.

The issue-tracker example will use a local read to correct a predicted observation. That isolates what feedback changes in the agent's belief and what it leaves unchanged in the predictor.