A team of researchers has introduced FM-VLA (Force-based Memory for Vision-Language-Action models), a novel approach that adds a dedicated force-based memory module to standard VLA architectures. Published on arXiv (2607.18231), the work enables humanoid robots to reason over sequences of physical contacts — unlocking capabilities in tasks where visual information alone is ambiguous.
The Core Innovation: Force Memory as Temporal State
The key insight behind FM-VLA is that robots can use the history of physical forces they've experienced as a form of memory, supplementing visual perception in contact-rich environments. The system uses a variational autoencoder (VAE) trained on sequences of 6-axis wrench signals (3-axis force + 3-axis torque) from wrist-mounted force-torque sensors.
The VAE learns a compact, task-agnostic representation of the robot's contact history, compressing a sliding window of wrench time series into just 8 latent tokens. These "force memory tokens" are then injected into the VLA's action expert transformer, allowing the policy to remember past physical interactions even when current visual observations are identical or ambiguous.
Impressive Results on Three Contact-Rich Tasks
Evaluated on an AgiBot G1 bimanual humanoid robot, FM-VLA was tested on three contact-rich tasks requiring temporal reasoning:
- Shell game (cup-and-block): 16–24 steps where the robot must remember which cups it has already lifted
- Peg insertion: sequential insertion where contact feedback guides alignment
- Drawer opening: manipulating handles that may appear visually similar
In 18 trials per task, FM-VLA achieved over 85% success rate across all tasks, dramatically outperforming baselines:
- Vision-only baseline (no force memory): less than 30% success
- Vision + single-frame force conditioning: ~50% success, often failing when visual cues were ambiguous
- Vision + vision-based memory (frame stacking): improved but still struggled with physically identical states
Notably, the force memory module adds less than 2ms of inference latency per step — negligible for real-time control.
Two-Stage Training Pipeline
FM-VLA uses a two-stage approach:
- Stage 1 — Force Memory VAE Pretraining: The VAE is trained on 100Hz wrench time series from all demonstration tasks combined, producing a general-purpose latent space that encodes temporal patterns of contact events.
- Stage 2 — Integration into VLA Policy: The frozen VAE encoder is attached alongside the existing visual and proprioception encoders. Force memory tokens are concatenated with action tokens in the action expert transformer, and the full policy is fine-tuned end-to-end on task demonstrations.
Ablation studies confirmed that removing the force memory or replacing it with proprioception-only signals reduced performance to baseline levels.
Why It Matters for Industrial Robotics
Force sensing is already cheap, robust, and widely available on humanoid and collaborative robots. FM-VLA demonstrates that adding a force memory module can unlock significant new capabilities without expensive hardware upgrades — a finding with direct implications for manufacturing, assembly, and logistics.
For industrial robots being repurposed for flexible automation, adding a low-cost force sensor and this memory module could enable new inspection and manipulation capabilities. In warehouse settings, force memory could track which packages have been handled even when barcode scanners or vision systems fail.
Limitations and Future Directions
The current implementation uses a fixed VAE bottleneck of 8 tokens, which may limit performance on very long-horizon tasks with hundreds of contact events. The researchers note that hierarchical or adaptive compression could address this.
Additionally, the VAE was trained solely on the three demonstration tasks. Pretraining on large-scale, diverse force-torque datasets could further improve generalization to new environments and sensor types.
The work was published as arXiv:2607.18231 in July 2026.
