Video generation models already absorb rich priors about how the visual world moves, deforms and responds to contact. The open question for robotics has been how to tell such a model what you want to happen. A team led by Stanford University, with the University of Maryland and Harvard — including Li Fei-Fei, Jiajun Wu, Wenlong Huang, Yilun Du and Jia-Bin Huang — submitted an answer to arXiv on July 21: Masked Visual Actions (MVA), a pixel-space control interface that turns a single video model into a unified world model for robots.
Action as a Partially Revealed Trajectory
The core idea is disarmingly simple. An action is expressed as a partially revealed trajectory of an arbitrary entity in a video — a mask that exposes where something moves over time. Reveal the robot's motion, and the model behaves as a forward dynamics model, predicting how the scene responds to low-level robot actions. Reveal the object's desired motion instead, and the same checkpoint runs inverse dynamics, synthesizing robot behavior consistent with that outcome. One interface, both directions of world modeling — no embodiment-specific action tokenization required.
15 Hours, One Checkpoint, Unseen Embodiments
The data efficiency is the headline number: the model was finetuned with only 15 hours of masked examples drawn from real videos and simulation. The result is a single checkpoint with strong visual fidelity and controllability across diverse scenes and multiple robot embodiments — and the team stresses that every embodiment shown on the project page was unseen during training, from tabletop arms to humanoid platforms. In a field where cross-embodiment transfer usually demands costly per-robot data collection, that is a meaningful claim.
Three Immediate Uses
The paper demonstrates three downstream applications that plug directly into existing robot-learning stacks. For policy evaluation, the model's imagined rollouts produce outcomes that correlate with real-world execution — a cheaper proxy for testing policies without wearing out hardware. For model-based planning, it ranks candidate futures and picks the action sequence whose imagined outcome looks best. And for inverse modeling, it generates robot motion from a desired object trajectory — effectively "show the goal, get the behavior."
MVA lands in the middle of the field's broader push toward world action models — the idea, articulated in several 2026 roadmap papers, that unified world models need a common, reusable way to express interventions. A pixel-space action interface is about as embodiment-agnostic as it gets. Code is available on the project's GitHub repository, which should make the approach easy to probe, benchmark and build on.
