Learning Realistic Expressions for Humanoid Face Robots
Yongji Fu, Rui Zhang, Zhenyu Xu, et al.
CVPR 2027, Submitted, 2025
For a humanoid face robot to appear approachable and believable in education, companionship, and performance settings, it needs to generate full-face dynamic expressions that engage the brows, eyes, lips, and cheeks. Existing systems typically treat the robot face as a downstream execution target of human facial animation, relying on low-dimensional expression parameters or pixel-video intermediates — which splits expression generation from physical execution: the former struggles to carry the diverse full-face motion that accompanies a single utterance, while the latter places the real robot under an out-of-distribution visual target and still requires extraction, smoothing, and mapping before it reaches the actuators. This work proposes a text- and audio-driven, high-DoF-density biomimetic face-robot system. On the hardware side, a flexible tendon-sheath transmission network decouples the servos from the facial actuation points, packing 34 servo actuators into a compact, approachable head only 19.5 cm tall. On the algorithm side, we learn a robot-aware motion latent interface and generate full-face motion directly in this space, so that text controls emotion and action semantics, audio constrains lip-sync, and robot image observations anchor the interface to the real hardware. The system avoids first generating pixel video or reconstructing the robot’s face at inference time, reducing intermediate artifacts and extra latency. Experiments show the system generates natural, controllable speaking expressions and provides a unified interface for robot expression driving that stays stable across performers.
@unpublished{fu2025humanoidface,
title={Learning Realistic Expressions for Humanoid Face Robots},
author={Fu, Yongji and others},
note={Manuscript in preparation; target venue: IEEE International Conference on Robotics and Automation (ICRA) 2026},
year={2025}
}