DW0.5: A Closed-Loop World Model for VLA

GitHub · Hugging Face

Background

Over the past three years, language models and agents have moved at a startling pace. Bigger models and larger datasets explain part of that progress, but not all of it. If scale were the whole story, why have code agents improved so much faster than agents in many other domains?

The key is a structural advantage: code is executable, and therefore self-verifiable. Embodied intelligence needs a similar path forward, but robotics still lacks the infrastructure that makes this loop cheap and repeatable.

The RL Loop Changed Post-Training

Large-scale pretraining gives models a foundation, but post-training feedback is what teaches them to act. The rise of code agents is a clear example. Code can be run, tests can judge correctness, and failed attempts can be converted directly into training signals: feedback is frequent, verification is explicit, and data naturally flows back into the model. RL and preference learning work unusually well for code agents because this closed-loop infrastructure exists, not simply because the models are larger.

Embodied intelligence needs the same kind of loop. A VLA model should do more than imitate expert actions from a dataset. It should try actions, observe what happens, estimate whether the task is moving toward success, and turn that feedback into a learning signal. Robotics does not yet have that native verification layer: no executable tests, no instant correctness signal, and every failed attempt carries a real physical cost.

Why the VLA Flywheel Stalls

A VLA outputs physical actions in the real world. Each step changes object positions, contact relationships, and the recoverability of the task. Each failure may require a manual reset, and every attempt consumes real time and hardware.

Real-robot rollouts are the core bottleneck. Robots are slow, equipment and maintenance are expensive, and repeated failures can damage hardware, making scale difficult. Human feedback is low-throughput and often too coarse for training. Robot tasks frequently need fine-grained judgments over intermediate states, such as whether a deviation is still recoverable. Asking people to label those signals continuously is prohibitively expensive. Traditional simulation still suffers from a sim-to-real gap, especially in everyday manipulation. Transparent and reflective objects, deformable materials, contact noise, and the long tail of home environments are all hard to model. Policies trained in such simulators often remain brittle on real hardware.

VLA needs a different kind of loop: cheaper than real robots, denser than human feedback, and closer to real physics than traditional simulation. That is the role of a world model: not a video generator bolted onto the system, but a learned environment for VLA post-training.

Approach: A World Model as the Training Environment

A VLA world model needs to answer three questions:

1. Given the current state and a candidate action, what happens next?
2. Does that future move the task closer to success, or farther away?
3. Can signals of success, failure, and deviation be turned continuously into VLA training data?

DW0.5 is built around these questions. It is not a video model optimized only for visual realism, nor a policy model that directly regresses actions from images and language. DW0.5 is intended to serve as a training environment for closed-loop VLA iteration: the VLA proposes candidate actions, DW0.5 simulates their consequences, the Value Expert scores those futures, and the training system uses the feedback for policy selection, data construction, or RL post-training.

The loop works as follows:

1. The VLA samples candidate actions from the language instruction, current observation, and robot state.
2. DW0.5 generates an action-conditioned future rollout for each candidate.
3. The Value Expert estimates success probability or task value for the current state, a candidate future, or the full rollout.
4. The system uses this feedback to select actions, build preference data, or run RL post-training for the VLA.
5. A small stream of real-robot rollouts keeps calibrating the world model toward the deployment distribution.

Real robots still matter. They are the starting point of the flywheel, just not the only training ground. Real rollouts calibrate the world model, the world model enables low-cost exploration at scale, and the improved policy returns to the real environment for validation and data collection.

DW0.5 uses three core designs to make this loop practical.

Design 1: Action as a Hard Condition, Not a Soft Prompt

A world model must pass a basic test before it can serve as a VLA training environment: from the same initial state, do different actions lead to different futures?

If pushing left and pushing right produce nearly identical rollouts, the "world model" is just a success-biased video generator. It cannot support meaningful VLA training. This is the central challenge in action-conditioned modeling.

Mapping actions into language space is one common approach: discretize continuous action sequences into tokens, add them to the language vocabulary, and feed them as conditions for video generation. It is simple, but it has two fundamental limits.

First, it is only a soft constraint. Once action tokens live in language space, they behave like ordinary semantic guidance. The model may use them, but it can also dilute them in favor of a visually coherent video. Give the model a failed action that pushes an object off course, and it may still generate a plausible-looking success. A training environment that must react precisely to different actions needs a stronger constraint.

Second, the action sequence is not temporally aligned with the video frames. In the physical world, actions are frame-by-frame execution commands. After projection into language space, that correspondence disappears. The model no longer knows where the end effector should be in frame 3, which frame should reflect gripper closure, or when contact occurs. Without frame-level alignment, action only influences generation at a broad semantic level rather than through the physical state of each frame.

Image-based conditions, such as end-effector trajectory projections, ray maps, or pixel-space guidance, are closer to control than text, but they add brittle dependencies. They typically require accurate camera intrinsics and extrinsics, robot geometry, and coordinate calibration. Change the camera, swap the tool, or mix data sources, and the system must be adapted again. More fundamentally, pixel projection is not the same as action semantics. It cannot fully express joint-angle changes, gripper timing, coupled multi-DoF motion, or the spatial meaning of the same action from different viewpoints.

DW0.5 instead treats action as a first-class condition for video generation and binds it with structured frame-level alignment. In MoT (Mixture of Tokens) self-attention, the action sequence and video sequence are concatenated into one token sequence, while a group-diagonal attention mask enforces frame-level constraints: video tokens for frame i can attend only to the i-th group of action tokens, rather than sharing all action information globally. This is not weight-level guidance. The attention mask structurally blocks information flow between video frames and non-corresponding actions. A left-push sequence and a right-push sequence take different computation paths from the start, producing different futures.

The goal is straightforward: the VLA samples multiple candidate actions, DW0.5 rolls out a distinct future for each one, and the Value Expert scores those futures. Only a world model that responds differently to different actions can function as an environment for exploring the VLA policy space.

Design 2: Simulating Both Success and Failure

To become a training environment, a world model must learn to simulate failure.

That may sound obvious, but it matters deeply. If the training data contains only expert success trajectories, the model learns a strong success prior. Whatever action it receives, it tends to generate a completed task. Such a model cannot warn the VLA that an action will push an object into an unrecoverable state, cannot penalize bad behavior, and cannot support RL post-training. It shows what success looks like, not what this action will cause. That is a video generator, not a simulator.

DW0.5's data strategy is built for this goal and draws from four sources:

Public embodied datasets and self-collected robot data provide manipulation experience across tasks, platforms, and views. The self-collected data records real execution under a unified sensor setup, including occlusion, contact, latency, jitter, and accumulated error. These are the kinds of noise that appear in deployment, not idealized simulation artifacts.

Internet video data adds open-world visual dynamics. Robot data cannot cover every object, scene, or event. Internet videos bring richer appearances, spatial layouts, and everyday dynamics, keeping the Video Expert from overfitting to the narrow distribution of robot-lab footage.

Egocentric data is easy to underestimate in this mixture. First-person human activity contains dense real-world object interactions: picking up, placing, opening, deforming, sliding, pouring, and correcting after failure. Humans and robots do not share the same action space, but object-level visual consequences and physical regularities do transfer. Ego data exposes the model to broader, denser physical interaction than narrow lab scenes alone, which is crucial for simulating contact outcomes, object motion, and deformation.

Real-robot and simulation rollout data is what makes DW0.5 behave like a simulator. We use successful and failed trajectories from routine collection and real-robot experiments, along with large-scale simulation rollouts that cover deviation, stuck states, recovery, and other intermediate conditions. We also add real success and failure cases from RoboChallenge. Simulation rollouts explore broader action spaces and failure boundaries efficiently, while real-robot rollouts contribute contact noise, execution error, and long-tail scenes. Together, they teach DW0.5 not only "what should happen," but also "what the world looks like when the action is wrong."

Simulating failure is a prerequisite for RL-based VLA post-training. Without it, there is no meaningful reward signal to optimize.

Below are successful and failed trajectories generated by DW0.5 for the same tasks:

Design 3: Turning Futures into Trainable Feedback

With strong action conditioning and rollouts that include failure, the world model can tell the VLA what a future may look like after an action. But that is not enough. RL post-training needs reward: it must know which actions to reinforce and which to suppress.

Rewards in robotics are often extremely sparse. Success or failure is usually confirmed only after the task is complete. Yet many failures reveal themselves much earlier: an object pose drifts away from the target, the gripper contact becomes unstable, the target is pushed into a hard-to-recover position, or the action moves the environment into a low-success region. If these signals arrive only at the end, training is highly inefficient.

DW0.5 introduces the Value Expert to turn generated futures into dense value signals. It estimates success probability or task value for the current state, a candidate trajectory, or the full rollout, converting sparse task outcomes into intermediate feedback that can be used at every step.

In the closed-loop system, the Value Expert supports:

• Candidate action filtering: scoring multiple rollouts so the VLA can choose the future most likely to succeed.
• Reward signals for RL post-training: optimizing the VLA policy inside the world-model environment without constantly using real robots.
• Online monitoring during deployment: detecting when the current state leaves the success path and triggering replanning or recovery in time.

The Value Expert is what moves DW0.5 from a "video world model" toward a training environment. The Video Expert simulates consequences; the Value Expert turns them into optimizable signals. Together, they form the core of an RL training loop.

One Loop, Three Roles

With these three designs, DW0.5 can play three roles in VLA training and deployment:

Offline data augmentation and preference construction. As the VLA learns expert actions from datasets, DW0.5 rolls out candidate futures and evaluates their value, automatically producing preference pairs, failure samples, and recovery samples for continual training.

RL post-training environment. The VLA samples action sequences inside DW0.5, observes generated future visual states, and receives reward feedback from the Value Expert, enabling low-cost policy optimization at scale without frequent real-robot use.

Planning and safety evaluation during deployment. Before execution, the system performs short-horizon rollouts for multiple candidate actions and selects the one with higher value and lower risk. If the Value Expert detects that the current state has left the success path, it triggers replanning immediately.

The goal is not to build a perfect simulator. It is to extract as much training value as possible from limited real-robot interaction: real rollouts calibrate the world model, the world model supports low-cost exploration at scale, and the improved policy returns to the real environment for validation and data collection.

Results

We evaluate DW0.5 on three complementary benchmarks. EWMBench and WorldArena measure world-model capabilities, while RoboTwin2.0 focuses on policy execution. DW0.5 achieves SOTA on all three, showing that it can generate future rollouts that can be evaluated while also producing stable action outputs.

World-Model Evaluation

EWMBench evaluates embodied world-model generation, focusing on scene consistency, motion correctness, and semantic alignment. DW0.5 ranks first with an overall score of 4.66.

*EWMBench overall ranking (as of 2026-07-09)*

WorldArena emphasizes open-world interaction dynamics and future-state modeling, measuring whether a model can predict plausible next states in complex scenes. DW0.5 ranks first with a score of 73.54.

*WorldArena overall ranking (as of 2026-07-09)*

RoboTwin2.0

DW0.5 reaches 93.3 in both clean and randomized settings, for an average score of 93.3, setting a new SOTA on RoboTwin2.0. RoboTwin2.0 directly evaluates action-policy capability in a dual-arm simulation environment. The result shows that the Action Expert remains stable across complex manipulation tasks and scene perturbations.

Demo

1. Across Environments and Tasks

2. Across Embodiments

3. Multi-View Consistency

4. Action-Video Consistency

In each example, the generated video is shown at the upper left, the initial frame at the lower left, and the action condition on the right.

Outlook

The next priority is broader generality. The current results show that DW0.5 already generalizes across tasks, robots, and scenes at a basic level. From here, the key is to expand coverage across robot morphologies, object properties, contact patterns, and long-tail environments, then consolidate those variations into a unified world representation. Only when that representation transfers reliably can the world model move from a rollout model that generalizes across scenes to a reusable learned environment.

As generality improves, the loop between the VLA and the world model should also become more efficient. The current pipeline of candidate actions, future rollouts, and value evaluation is already in place, but more complex tasks will require tighter coordination among VLA sampling, world-model prediction, Value Expert scoring, and real-robot calibration. The smoother this loop becomes, the better it can support high-cost, long-horizon, hard-to-reset manipulation tasks, and the more the world model can move beyond offline data augmentation into policy exploration, failure recovery, and pre-deployment validation.

Looking further ahead, the RL framework itself needs a stronger closed loop. Many robot RL systems still rely on DAgger, human annotation, or human preferences for feedback, which is expensive and hard to scale. With the Value Expert, rollouts can already be converted into value estimates and rewards, but that is only the beginning. A fuller loop should let the model discover task progress, early failure signals, and trajectory preferences during continuous rollout, then turn them into iteratively updated reward and preference data, so that "act, predict, evaluate, and retrain" can start to run in practice.

---

The VLA post-training flywheel needs a world model that can simulate action consequences, include failure cases, and provide optimizable reward signals. DW0.5 is a first step in that direction.