Taxonomies of World Models › Task-Specific and General-Purpose Models

The final axis concerns scope. Most world models built to date are task-specific: trained on data from one environment, often with that environment's reward signal entangled in the training objective, and discarded when the task changes. Dreamer agents are trained per game; identified dynamics models describe one robot arm; the pendulum model we build below describes one pendulum. Task specificity is not a defect so much as a consequence of economics: collecting broad interaction data is expensive, and if your decision problem lives in one environment, generality beyond it is wasted capacity.

General-purpose world models aim to break that coupling. The ambition, visible across recent research lines, is a single model trained on internet-scale video and interaction data that captures broadly reusable dynamics: objects persist, liquids pour, people walk, gravity pulls downward. Systems in this spirit include large video generation models such as Sora, interactive world generators such as Genie (Bruce et al., 2024), and industrial efforts such as NVIDIA's Cosmos, which explicitly targets physical AI applications. LeCun's JEPA proposal argues that the path runs through latent prediction of abstract representations rather than pixel generation. Note how each of these programs occupies a different position on the earlier axes: Sora-style models are explicit and pixel-scale, Genie is explicit over latent action-conditioned dynamics, and JEPA advocates latents with deliberately suppressed pixel detail. The generality debate is thus inseparable from the representation debate, and progress on one axis reshapes the options on the other.

Moving rightward along this axis changes far more than dataset size:

• Action conditioning becomes the hard part. Passive video teaches how the world evolves on its own; agents need "what happens if I do X," and internet video rarely contains reliable action labels. Inferring or annotating actions, or learning latent action spaces as Genie does, is an active research frontier we revisit in Part IX. The difficulty is deeper than labeling: much of what makes video useful for prediction is precisely that it was produced by someone's unrecorded intentions, and recovering those intentions from pixels alone is an inverse problem with no unique solution.
• Evaluation changes character. A task-specific model is judged by the performance of the agent it powers. A general-purpose model needs benchmarks for physical plausibility, controllability, and consistency across contexts, the concerns of Part XI. There is currently no consensus equivalent of "win rate" for general world models, which slows progress in ways that raw compute cannot fix.
• Failure modes scale up. A task-specific model that hallucinates wastes one training run. A general-purpose model that hallucinates convincingly, at scale, becomes an infrastructure risk wherever downstream agents trust it. Convincingness and correctness decouple at scale: a model trained to produce plausible video has every incentive to render physically impossible events beautifully, and consumers who judge by appearance will be misled systematically rather than occasionally.

It is worth resisting the temptation to treat this axis as a progress ladder with general-purpose models at the top. Task-specific models remain the right answer whenever you have a fixed environment, a modest data budget, and a concrete decision problem. Generality is expensive, and its value materializes only when amortized across many downstream uses. The correct framing is amortization: a general-purpose model is a fixed cost spread over an uncertain number of future tasks, and it wins exactly when that number turns out large. Betting on it is a strategy decision, not a scientific inevitability.