Smart Speaker Voice Interaction Technology: Recent Advances and Tencent's Research Progress

This article presents a comprehensive overview of smart speaker voice interaction technology, based on a分享 by Tencent researcher 王珺. The content covers the complete technology chain for smart speaker interaction, including five key modules: voice capture and enhancement, speaker verification, speech recognition, semantic understanding, and speech synthesis.

Front-end Voice Processing: The article discusses the AIVP system that integrates voice detection, sound source localization, microphone array beamforming, directional pickup, noise suppression, reverb elimination, and automatic gain control. For voice wake-up technology, the author addresses challenges including false wake-up, noise robustness, fast speech wake-up, and child voice wake-up. Through algorithm upgrades and model compression, the team achieved over 60% reduction in false wake-up rate.

Speaker Verification: The article explains voiceprint recognition technology for identifying users based on voice characteristics, enabling personalized applications for different family members. The system can also determine user gender and age for targeted recommendations. Challenges discussed include channel mismatch, environmental noise, short speech, and far-field scenarios. The team developed new algorithms for short-speech voiceprint recognition that outperform mainstream approaches.

Speech Recognition: The article covers TDNN and TDNN+LSTM network structures, CTC and Attention training frameworks, and personalized speaker-adaptive models that improved recognition accuracy by 2%. For Chinese-English mixed recognition, the team achieved 90% English recognition accuracy while maintaining Chinese performance. The language model uses a 100-billion n-gram model with LSTM+adaptive-softmax for fast rescore.

Target Speaker Extraction: A novel dual embedding space mapping approach is proposed for extracting target speaker voice from mixed audio. This method addresses limitations of existing technologies that require long adaptation speech and have poor extensibility. The approach uses LSTM networks to compute embedding vectors and relative position information between embedding spaces for reliable speaker separation.

End-to-End Attention Modeling: The article presents关键技术点 for combining CTC and Attention methods, including Minimum Bayes Risk (MBR) loss, softmax smoothing for N-best generation, decoder feedback input structure, and attention transformation layers. These techniques enable the system to achieve competitive results without external language models.