Vision-Language-Action and World-Action Models › Action Tokenization and Robot Policies

The action space is where VLA models differ most from vision-language models, and where consequential engineering decisions get made. Language models emit discrete symbols from a finite vocabulary. Robot actions are continuous vectors, often 7 to 30 dimensional, sampled at 10 to 100 Hz. Bridging that gap is the subject of this section. Representation choice determines the loss function, inference procedure, achievable command precision, and action distributions the model can learn.

Digital images and tokenized text have discrete input encodings, though pixel bit depths vary and a word need not correspond to one token. Robot actions are often represented as continuous vectors with physical units. An autoregressive token decoder needs a discrete action encoding; a transformer with a continuous output head does not. Three common action-output choices are:

• Discretize each dimension into bins and predict a categorical distribution, optionally folding the bins into the language vocabulary as tokens.
• Regress directly, with a mean-squared-error or L1 loss on a continuous output head.
• Generate by sampling from a learned distribution over continuous actions, using diffusion or flow matching.

These choices are not exhaustive: joint categorical distributions, learned codebooks, mixtures, and other heads are possible. The selected output representation shapes the training objective, inference cost, and which action distributions the model can express.

The simplest scheme maps each action dimension from its operating range $[\ell_d, u_d]$ into $B$ equally spaced bins, then predicts a categorical distribution over bins. Encoding is

$$\text{bin}(a_d) = \operatorname{clip}_{[0,B-1]}\!\left(\operatorname{round}\!\left( \frac{a_d - \ell_d}{u_d - \ell_d} (B - 1) \right)\right)$$

where:

• $a_d$: the continuous value of the $d$-th action dimension, in the operating range $[\ell_d, u_d]$
• $\ell_d$: the lower bound of the operating range for dimension $d$
• $u_d$: the upper bound of the operating range for dimension $d$
• $B$: the number of equally spaced reconstruction values, with $B \geq 2$ and $u_d > \ell_d$
• $\text{bin}(a_d)$: the integer bin index assigned to $a_d$, in $\{0, \dots, B - 1\}$

For the displayed endpoint-grid encoding, decode index $k$ as $\ell_d + k(u_d-\ell_d)/(B-1)$. Adjacent reconstruction values are $(u_d-\ell_d)/(B-1)$ apart, so rounding gives a maximum absolute error of $(u_d-\ell_d)/(2(B-1))$ for inputs inside the stated range. A distinct midpoint-bin convention partitions the range into $B$ intervals and decodes to each interval's center; its maximum error is $(u_d-\ell_d)/(2B)$. These conventions require different encoders and must not be mixed. For $[-1,1]$ and $B=256$, the endpoint grid has spacing $2/255 \approx 0.00784$ and maximum error $1/255 \approx 0.00392$ normalized units. If one normalized unit corresponds to one radian, that bound is about 3.92 milliradians; whether it is below a controller's effective resolution depends on the hardware.

Two properties of this scheme deserve attention.

The first is that the distribution of actions depends on the dataset and control convention. If demonstrations concentrate near zero or near saturation, uniform bins may spend resolution on rarely used regions; another dataset may use the range more evenly. Inspect the target distribution before choosing a grid. Per-dimension normalization using training-set statistics or percentiles can reduce scale mismatch but does not make the distribution uniform. Nonuniform bins can allocate more values to dense regions, at the cost of a less simple encoding and decoder.

The second is that discretization is not a lossless stand-in for regression. With a fixed endpoint grid, each decoded command must be one of its reconstruction values, with the per-command quantization bound derived above. That does not impose the same bound on final position: feedback, repeated relative commands, and dynamics can yield intermediate endpoints. Tasks needing finer individual commands can use more bins, a smaller action scale, or a continuous head.

Turning bins into language tokens requires one extra decision: which tokens. Repurposing tokens needed for ordinary text can interfere with the model's language behavior, so the mapping depends on the pretrained tokenizer and on how action outputs are decoded.

RT-2 used different mappings for its two backbones. PaLI-X already had tokens for the integers 0–999, so its action bins reused those numeric tokens. PaLM-E lacked that convenient numeric representation, so RT-2 overwrote its 256 least frequently used tokens for the action vocabulary. During robot-action decoding, RT-2 restricts sampling to valid action tokens; on ordinary vision-language tasks, the full language vocabulary remains available. The task context and decoding rule, not a universally private token namespace, determine how an output is interpreted.

The elegance comes with a cost, and the cost is sequence length. The token stream that carries the actions is much longer than the geometric action chunk it represents, and that length is what determines inference latency.

Consider a seven-joint arm with one additional gripper command, controlled at 50 Hz. Each action vector has eight scalars, so one second of behavior contains 400 scalars; with one token per scalar, that is 400 tokens per second of behavior. A seven-billion-parameter autoregressive model may decode on the order of 50 to 150 tokens per second in a particular serving setup, depending strongly on hardware, precision, batching, and implementation. At those assumed rates, generating one second of behavior takes about 2.7 to 8 seconds. This setup therefore cannot sustain 50 Hz through one-token-at-a-time decoding. Faster hardware can narrow the gap, but it does not remove the sequential decoding steps required by this representation.

Action chunking is the first fix. If a non-autoregressive head emits a 50-step chunk in one call, the controller can query it once per second instead of fifty times. An autoregressive language decoder still needs a decoding step for each token: chunking alone leaves the 400-token count per second of behavior unchanged. It can reduce policy or control-interface calls, but it does not remove the underlying next-token decoding cost.

The second fix attacks the token count directly. Smooth robot trajectories often concentrate more of their energy in low-frequency coefficients along the time axis. Quantizing those coefficients and then applying byte-pair encoding to the resulting symbol stream can compress an action chunk. We use an illustrative 400-scalar-to-30-token ratio in the worked example below, rather than treating that ratio as universal. At 30 tokens per second of behavior, the representation fits within the assumed 50 to 150 token-per-second decoding throughput. FAST exploits this temporal regularity in a way analogous to how image compression exploits spatial regularity.

The third option uses a continuous action head rather than per-scalar tokens. A regression head can emit a chunk in one evaluation. Diffusion and flow heads generate a chunk through an iterative solver whose evaluation count depends on the model and solver; some implementations use about ten steps. These heads avoid per-scalar autoregressive decoding, but only stochastic generative heads model multimodality without an added mixture mechanism.

A regression head adds a linear layer mapping the backbone's pooled representation to an action vector, trained with $L_2$ or $L_1$ loss. It is the cheapest option and the most limited. It has no mechanism to express uncertainty, no way to represent more than one answer per input, and no generative sampling procedure.

The limitation is mode averaging. Demonstration data can be multimodal: a teleoperator may go around an obstacle on either side, and both are correct. For an unconstrained predictor minimizing expected squared error, the optimum is the conditional mean, which may be valid in neither mode. For scalar $L_1$ loss, any conditional median is optimal; it may lie in a dominant mode, but for equally weighted, separated modes a median can also lie between them. Either deterministic regression head still represents only one action per input. Vector-valued $L_1$ loss yields coordinatewise medians, which need not form a feasible joint action; the coordinatewise medians of a joint distribution can lie in a region the joint distribution assigns essentially no probability.

Diffusion and flow-matching heads can address mode averaging by learning a conditional distribution from which multiple action chunks can be sampled. They do not guarantee that the fitted distribution captures every valid mode. Flow matching, used in the $\pi_0$ family, defines an interpolant between noise $\epsilon \sim \mathcal{N}(0, I)$ and a target action chunk $a$,

$$x_\tau = (1 - \tau)\,\epsilon + \tau\, a, \quad \epsilon \sim \mathcal{N}(0, I), \quad \tau \in [0, 1],$$

where:

• $\epsilon$: a noise sample drawn from a standard normal, $\mathcal{N}(0, I)$, with the same shape as the action chunk
• $a$: the target action chunk from the demonstration data
• $\tau$: the interpolation time, running from $0$ (pure noise) to $1$ (clean action)
• $x_\tau$: the interpolated sample at time $\tau$, the input the velocity field is asked to explain

The backbone context $c$ encodes the observation and instruction. A velocity field $v_\theta(x_\tau, \tau, c)$ is trained to regress the straight-line target velocity $a-\epsilon$.

$$\mathcal{L}_{\text{FM}}(\theta) = \mathbb{E}_{\tau \sim \mathcal{U}[0,1],\, \epsilon \sim \mathcal{N}(0,I),\, (a, c) \sim \mathcal{D}} \left[ \left\| v_\theta(x_\tau, \tau, c) - (a - \epsilon) \right\|^2 \right]$$

where:

• $v_\theta(x_\tau, \tau, c)$: the velocity field, parameterized by $\theta$, evaluated at the interpolated sample $x_\tau$ at time $\tau$ with context $c$
• $a - \epsilon$: the constant velocity along the straight-line interpolation from noise to data, which is what the field is regressed toward
• $c$: the backbone's context, which conditions the field on the observation and instruction
• $\theta$: the learnable parameters of the velocity field

The taxonomy in Table action-heads uses a factorized per-dimension categorical head. Its dimensions are sampled independently, so a bimodal task can combine the upper $x$ mode with the lower $y$ mode and produce a joint action absent from demonstrations. Correct marginals do not guarantee a valid joint sample. A joint or autoregressive categorical model need not have that independence, but it pays for dependency modeling in complexity or decoding steps. Chunk-level tokenizers such as FAST offer another way to encode cross-time and cross-dimension structure; which representation works better requires task-specific evaluation.

Three implementation choices can affect whether a heterogeneous VLA trains reliably. They are easy to miss in a high-level architecture diagram and worth checking explicitly.

Per-dimension normalization. Action dimensions can have different scales and units. Shoulder rotation in radians and gripper opening in meters can share an optimizer, but unnormalized MSE gives the numerically larger dimension much more weight: compare ranges $[-3, 3]$ and $[0, 0.05]$. Many VLA pipelines therefore normalize action dimensions using training-set statistics or quantiles; the chosen convention must be saved and inverted at execution time.

Absolute versus relative actions. Absolute joint positions depend on a specific kinematic calibration. Relative commands, such as an end-effector pose delta plus gripper command, can be easier to reuse across similar arms, but their physical effect still depends on the body and controller. A controller can apply a delta using the current measured pose; the policy need not integrate every previous delta internally if proprioception provides that pose.

Padding and masking. Heterogeneous datasets may have different action dimensionalities. One approach pads actions to a shared maximum width and masks padded entries in the loss; another maps commands into a canonical action space, as in RT-X. If padded entries are left unmasked, the loss also trains the model to predict filler values that are not real commands. The choice must be documented alongside action units and decoding.