How HumDex Overcomes Humanoid Robot Data Bottlenecks with Low‑Cost Full‑Body Dexterous Control

HumDex System Overview

Full‑body dexterous manipulation for humanoid robots requires coordinated control of high‑DOF arms, multi‑finger hands, and whole‑body poses. Acquiring high‑quality demonstration data is a bottleneck. Traditional teleoperation faces a portability‑vs‑precision trade‑off: optical motion‑capture or exoskeleton systems provide accurate tracking but need fixed infrastructure; VR‑based portable systems suffer occlusion and lack fine finger control.

All‑Scene Portable Tracking

HumDex replaces external cameras with a fully wireless inertial motion‑capture suite. Operators wear lightweight body‑tracking nodes and an inertial data glove that directly measures finger pose. The glove maintains high‑resolution finger and full‑body tracking even under severe visual occlusion. Compatibility with the open‑source SlimeVR ecosystem keeps the body‑tracking hardware (excluding the glove) under $200 while achieving data‑collection success rates comparable to commercial systems.

Learning‑Based Hand Redirection

Controlling a 20‑DOF dexterous hand traditionally uses optimization‑based inverse kinematics, which is computationally heavy, requires per‑user parameter tuning, and often produces finger coupling or stiff motions. HumDex trains a lightweight multilayer perceptron that maps 3‑D fingertip positions from the glove to the robot hand’s 20 joint angles. Training uses less than 20 minutes of paired motion data; inference runs in constant time, eliminating manual tuning.

Two‑Stage Cross‑Embodiment Training

HumDex first pre‑trains an ACT policy on large‑scale human demonstration data, learning generic visual and motion priors. A second fine‑tuning stage uses a small amount of robot tele‑operation data to align these priors with the robot’s embodiment, addressing the dynamics and kinematic gap. Experiments show that naïvely mixing human and robot data reduces task success to near zero, whereas the two‑stage approach restores high performance.

Real‑World Experimental Evaluation

1. Long‑Horizon Tasks and Tele‑Operation Efficiency

HumDex was evaluated on barcode packing, hanging clothes, opening doors, and basket grasping—tasks that involve fine pinch, dual‑arm coordination, flexible object handling, and whole‑body movement. Compared with a mainstream VR‑based tele‑operation system, HumDex reduced data‑collection time by 26 % and increased remote‑operation success from 74.6 % to 91.7 %. Policies trained on HumDex‑collected data achieved an autonomous success rate of 80 %, versus a baseline of 57.5 %.

2. Hand Redirection Comparison

Qualitative tests show the optimization‑based method frequently suffers finger coupling or failure on precise pinching, while the learning‑based method produces smooth, stable tracking with reliable single‑finger control. Quantitatively, across three fine‑contact sub‑tasks, the learning‑based redirection consistently improves remote‑operation success regardless of whether the underlying tracker is inertial or VR‑based.

3. Zero‑Shot Generalization via Human Data

In a bread‑grasping task, the two‑stage pipeline was tested on out‑of‑distribution scenarios: unseen object positions, novel objects (apple, banana, leaf), and new backgrounds (different tablecloth colors). Strategies trained only on robot data degraded sharply; the two‑stage approach nearly doubled success rates across all variations, demonstrating that human data provides transferable visual and motion priors that broaden policy generalization.

References

Paper: HumDex: Humanoid Dexterous Manipulation Made Easy – https://psi-lab.ai/humdex

GitHub repository: https://github.com/physical-superintelligence-lab/humdex