Google's i‑S2R and GoalsEye: Robot Table‑Tennis Learning from Human Interaction

Robotic table‑tennis serves as a compelling benchmark for human‑robot interaction because it demands high speed, precision, and coordinated multi‑agent collaboration, making it an ideal platform for studying reinforcement learning and real‑time control.

While robots possess dexterous manipulators and agile locomotion, the rapid dynamics of a ping‑pong ball pose significant algorithmic challenges, especially when interacting closely with humans.

Google's robot research team built two platforms—Iterative‑Sim2Real (i‑S2R) and GoalsEye—to investigate how robots can learn from and cooperate with human players in this structured yet dynamic environment.

i‑S2R employs an iterative loop between simulation and real‑world deployment, using a simple human‑behavior model to bootstrap learning; the system can execute over 300 rallies with a human, achieving up to 340 consecutive hits.

Experimental results show that for beginner and intermediate players (80% of participants), i‑S2R outperforms the baseline Sim‑to‑Real plus fine‑tuning (S2R+FT) strategy.

GoalsEye focuses on precise target hitting, leveraging behavior cloning, Learning‑from‑Play (LFP) and Goal‑Conditioned Supervised Learning (GCSL) to direct the ball to specified locations on the table, improving hit‑accuracy from 9% to 43% after thousands of demonstrations.

Both projects are publicly available for further exploration at the following URLs:

Iterative‑Sim2Real homepage: https://sites.google.com/view/is2r

GoalsEye homepage: https://sites.google.com/view/goals-eye