Understanding and Mitigating Hallucinations in Large Language Model Industry Q&A with Knowledge Graphs

The presentation begins by highlighting the common practice of using large models for industry Q&A and the frequent poor performance due to hallucinations, emphasizing the need for reliable answers and confidence scores.

It outlines five main topics: (1) implementation and challenges of large‑model industry Q&A, (2) definition, sources, and evaluation of hallucinations, (3) real‑world issues in document Q&A, (4) strategies to alleviate hallucinations, and (5) a summary.

Key challenges include complex document layouts, model over‑confidence, domain‑specific embedding noise, and the tendency of models to focus on the beginning and end of long texts, leading to "lost in middle" problems.

Hallucinations are defined as answers that conflict with context or factual knowledge, illustrated with examples from security and historical queries. Their origins are traced to noisy pre‑training data, misaligned instruction fine‑tuning, and decoding strategies such as top‑k/top‑p.

Evaluation methods discussed include truthfulness benchmarks (e.g., TruthfulQA), NLI‑style fact consistency checks, and overlap analysis of answer pairs.

Four mitigation strategies are proposed:

Construct high‑quality fine‑tuning data and incorporate a refusal module that answers "I don't know" when uncertain.

Introduce honest alignment during reinforcement learning to encourage truthful responses.

Adopt advanced decoding techniques like Context‑Aware Decoding (CAD), KNN+LLM fusion, and Retrieval‑Augmented Language Modeling (RALM).

Enhance with external knowledge bases or knowledge graphs, using single‑pass or iterative retrieval‑augmented generation to verify and correct answers.

The article concludes with a Q&A session covering embedding usage, hallucination impact percentages, model compression effects, knowledge‑graph integration, and threshold selection for similarity search.

Overall, the content provides a comprehensive technical overview of hallucination problems in large‑model deployments and actionable solutions for practitioners.