How Dual-Arm Humanoid Robots Collaborate Seamlessly: Tencent’s New Framework

Overview

Human‑like robots are rapidly advancing, and Tencent’s Robotics X lab has released a new dual‑arm framework that addresses perception, grasping, collaborative planning, and low‑level control.

Key Challenges

Three common challenges are identified: (1) vision‑based optimal dual‑arm grasping of unknown objects, (2) ensuring safety when the workspaces of two arms heavily overlap, and (3) transferring rich human skills to robots with limited tracking interfaces.

Proposed Architecture

The architecture consists of two tightly coupled sub‑systems.

Learning‑based dexterous reachability perception uses an end‑to‑end evaluation network and probabilistic modeling of robot reachability to achieve optimal cooperative grasping of unmodeled objects.

Optimization‑based multifunctional control adopts a hierarchical multi‑priority optimization framework, embedding a lightweight distance proxy generated by learning and a Riemannian‑manifold velocity tracking controller. This enables high‑precision dual‑arm collision avoidance and realistic dual‑arm operation‑ellipsoid tracking, guaranteeing intrinsic safety and exposing an ellipsoid‑tracking interface for skill transfer.

Demonstrated Capabilities

The framework enables a variety of differentiated tasks, illustrated below.

Dual‑hand coordinated screw turning

Dual‑hand coordinated pouring

Interference suppression

Human‑robot physical collaboration

Dynamic interaction based on object affordance and intent recognition

Autonomous handover of large objects

Impact and Hardware

The work bridges AI and robotics, improving the stability and reliability of AI‑generated strategies in the physical world and accelerating the path toward fully autonomous, general‑purpose dual‑arm systems. Tencent’s lab also introduced the self‑developed “TRX‑Hand” dexterous hand and the seven‑degree‑of‑freedom “TRX‑Arm”, as well as multimodal quadruped “Max” and wheeled‑leg robot “Ollie”. The study was published in IEEE Transactions on Robotics.