Limitations of Language Models in Voice Interaction and HomeAI Solutions

iQIYI HomeAI serves multiple platforms and products within iQIYI, providing a video‑on‑demand‑centered intelligent voice interaction solution. It continuously innovates by leveraging the latest speech‑related technologies to create new user experiences.

In most voice interaction systems, speech recognition, semantic understanding, and action execution are independent modules. Errors from the speech recognizer are invisible to downstream modules, causing error amplification and ultimately incorrect results. As intelligent voice assistants cover more domains, deficiencies in language models become a bottleneck for the entire system.

Limitations of Language Models

Statistical language models, learned from large text corpora, are widely used in speech recognition and NLP because they are more robust than rule‑based models. However, they cannot quickly adapt to hot topics or new TV shows, making it difficult for speech‑recognition and intent‑recognition modules to respond promptly. Moreover, the language model used for video‑library search evolves rapidly, creating a mismatch that prevents the recognizer from favoring entities that match the library content.

A Typical Error

When a user says “声临其境”, the phrase is absent from the recognizer’s language model, but a phonetically similar idiom exists. The recognizer outputs a high‑confidence wrong result, leading to an unexpected outcome (Path 1). Post‑processing such as fuzzy matching (Path 2) yields low‑confidence results that can be overridden by other domains (Path 3). Even providing the top‑N ASR hypotheses does not guarantee the correct result because of the language‑model defect.

HomeAI addresses this by correcting ASR output at the intent‑recognition layer using its own language model (Path 4), thereby reducing error propagation.

Separating the Language Model

To mitigate the impact of the ASR language model on entity recognition, HomeAI decouples phonetic information from semantic information. The decoder first outputs pinyin (pronunciation) and then converts it to characters, preserving both layers for downstream use. External ASR services are also processed to extract pinyin for entity retrieval, reducing the influence of ASR errors on intent recognition.

Entity‑Enhanced Intent Recognition

HomeAI follows a domain → intent → slot‑filling pipeline. Because phoneme‑to‑text conversion is weakened, intent recognition must be reinforced. HomeAI performs entity correction in two steps:

1. Combine ASR semantic and acoustic outputs with the intent‑recognition language model to correct entities in the original hypothesis.

2. Concatenate pinyin features with word embeddings, improving the model’s ability to generalize over similar pronunciations.

Conclusion

The lack of synchronized language models across modules and the absence of backward feedback cause speech‑recognition errors to be amplified downstream, leading to unsatisfactory user experiences. By weakening the semantic conversion in ASR and delegating entity enhancement to the intent‑recognition stage, HomeAI significantly improves user experience in video‑on‑demand scenarios.

Future Development

HomeAI will continue to invest in voice interaction research, expanding beyond video‑on‑demand to more adaptive scenarios. Future work includes building user‑ and context‑adaptive interaction models that can gradually learn a user’s accent, habits, and relevant entities through feedback, moving toward a personalized voice assistant.