Carnegie Mellon University researchers have developed Touch Dreaming (HTD), a breakthrough framework that gives humanoid robots the ability to imagine future tactile sensations.
The core innovation is a multi-modal Transformer architecture integrating vision, tactile sensing, and proprioception. Using a teacher-student approach with EMA teacher network for latent tactile targets, the student learns to predict future tactile signals alongside actions and forces.
Tested on five real-world tasks including precision insertion (3.5mm gap), towel folding, and scooping, HTD achieved 90.9% relative improvement over baselines. Zero-shot sim-to-real transfer was also demonstrated.
This represents a significant step toward humanoid robots capable of fine manipulation tasks from assembly to household chores.
