NLP Technology Applications and Research in Voice Assistants

Introduction : The talk titled “NLP Technology Applications and Research in Voice Assistants” introduces three main topics: Conversational AI Agent, XiaoAI Model Pipeline, and Self‑Learning.

1. Voice Conversation Process : Describes the typical pipeline where a user's speech is processed by an ASR system to produce 1‑best or n‑best text, which is then fed to an NLU module for intent and slot detection. A Dialog State Tracking (DST) component maintains slot inheritance across turns, followed by a ranking step to select the best skill, execution of the skill, and finally a TTS module that converts the response text back to speech.

2. NLU Example : Shows a domain classification step that routes the input to multiple verticals (e.g., music, video). Within each vertical, intent recognition, slot extraction, DST, and entity resolution are performed, and the results are ranked to produce the final intent and slot output. Multi‑turn examples illustrate how the system handles clarifications and corrections.

3. Intent and Slot Model : Introduces a knowledge‑enhanced model that combines a dedicated knowledge encoder with a pretrained BERT encoder. Features from both encoders are fused, and a multi‑task head predicts intent (via Softmax) and slots (via CRF). The model incorporates domain‑specific knowledge such as song titles and artist names, and also uses phoneme information to mitigate ASR errors.

4. Multi‑turn Dialogue Interaction : Describes the Dialog Act framework, distinguishing User Dialog Act and Agent Dialog Act. An example of a phone‑call scenario shows how the system asks clarification questions, inherits slots across turns, and confirms actions. A simulator generates synthetic dialogues by combining predefined dialog flows with noise (ASR errors, user corrections).

5. XiaoAI Model Pipeline : The pipeline consists of three parts: Data Platform, Model Toolkit, and Server Framework. The Data Platform aggregates public corpora, online logs, and labeled data for pre‑training and fine‑tuning. The Model Toolkit provides common layers (CNN, Transformer, RNN), model zoo (BERT, ALBERT), and task‑specific heads for intent detection, slot filling, etc. The Server Framework handles online and offline services, model compression, and deployment.

6. CentraBert : Describes a flexible BERT serving architecture where different tasks use different numbers of fine‑tuned layers. Knowledge distillation reduces layer depth per task, and the resulting multi‑task model serves only the necessary layers for each vertical, greatly reducing latency and QPS.

7. Quantization Techniques : Explains weight and activation quantization (int8), scaling factors, symmetric and asymmetric quantization, cross‑layer equalization, and post‑training quantization workflow. Shows how quantized models retain accuracy while reducing hardware costs.

8. Self‑Learning : Defines error types (wake‑up, ASR, NLU, execution) and introduces feedback learning from explicit (user corrections) and implicit (behavioral) signals. Describes a query‑rewriting model that corrects ASR/NLU errors and a correction model that detects and masks errors before rewriting.

Q&A Highlights : Answers cover annotation cost for dialog acts, rule‑based vs model‑based dialog management, handling ASR uncertainty with confidence scores, ranking‑based disambiguation, cross‑domain slot inheritance, and integration of task‑oriented and chit‑chat dialogues.