Embodied world models (EWMs) — video-generation systems that predict how a scene responds to robot actions — are multiplying fast. Benchmarking them properly is not. A paper submitted to arXiv on July 22 and accepted to ECCV 2026 takes aim at the weakest link in today's evaluation pipelines: the inverse dynamics model (IDM).
The Attribution Problem Nobody Wanted to Own
The standard closed-loop evaluation recipe goes: let the world model imagine a video rollout, extract the implied actions with a learned IDM, execute those actions in a simulator, score the outcome. The catch is structural. Mapping 2D pixels back to 3D kinematic actions is intricate, and learned IDMs turn brittle the moment generated videos contain objects or scenarios outside their training distribution. When a rollout fails, you can no longer tell whether the world model hallucinated bad physics or the extractor mangled good ones — an attribution ambiguity that quietly corrupts every leaderboard built on this pattern.
KineBench's Answer: Skip the IDM Entirely
KineBench replaces the learned extractor with an explicit kinematic grounding pipeline. Cascaded visual foundation models read each generated frame and directly recover 6D end-effector poses — no action inference required. Those poses are then executed in a physics simulator for closed-loop validation, so the benchmark measures what actually matters: whether the imagined motion is executable. Scoring goes beyond binary task success, adding two classic robot-centric kinematic metrics — SPARC (Spectral Arc Length) for trajectory smoothness and the Maruyama Manipulability Index for kinematic feasibility — so models are graded on motion quality, not just outcomes.
20 Tasks, Four Difficulty Ladders, One Uncomfortable Finding
The benchmark is built on 20 diverse manipulation tasks in ManiSkill3, organized into four progressive suites: basic execution, task transfer, visual out-of-distribution generalization, and complexity-conditioned scaling. Run across frontier EWMs, the evaluation surfaces a finding with direct consequences for the field's data-scaling race: embodied video generation exhibits task-complexity-bounded nonlinear scaling — capabilities do not improve smoothly with more data, and the ceiling depends on how hard the task is. For teams planning million-hour video-training budgets, that is empirical guidance money usually cannot buy.
KineBench lands as the world-model stack is becoming a load-bearing layer of robot learning — from policy evaluation to model-based planning. An IDM-free, execution-grounded yardstick gives the community a cleaner signal on which models are ready to plan, and which are just dreaming in high resolution.
