Roundtable on Enhancing Large Model Effectiveness: RAG, Tool Use, and Knowledge Engineering

The article records the discussion of the Data Intelligence Knowledge Map 3.0 release roundtable, focusing on how to boost large model effectiveness and control hallucinations, covering topics such as Retrieval‑Augmented Generation (RAG), agent planning, multimodal approaches, fine‑tuning methods like PPO and DPO, and cost‑effective deep‑seek techniques.

Participants included Huang Rongping (Manager, Dipu Model Engineering), Li Jiannan (Head of Chat DBA R&D, iKang), Qi Xiang (Head of NLP Algorithms, Ant Financial), and Zhang Yafeng (Head of Intelligent Algorithms, Zhihu).

Huang presented three case studies: a manufacturing AI‑assisted engineering design Q&A system combining LLM and RAG for fast, accurate information retrieval; a medical AI guidance system built by fine‑tuning a large model to assist patients and doctors; and a retail private‑model solution that generates data analysis reports for any store within ten seconds.

Host Li Jiannan asked about differences in fine‑tuning across scenarios; Huang explained that variations stem from distinct customer needs, which dictate the training direction, data preparation, and task objectives for each industry.

Qi Xiang described Ant Financial’s B2B focus, emphasizing two directions—RAG and Tool Using. He highlighted the importance of offline knowledge engineering (the “t + 1” mode) to pre‑process documents, generate FAQs, and simplify online inference. He also mentioned research on aligning large models to follow retrieved knowledge via reinforcement‑learning alignment, and evaluation methods using CFQA data and RAG‑AS scoring, extending assessments to tool‑using and BI scenarios with attribution modules.

Zhang Yafeng shared Zhihu’s C‑end advancements: modular drag‑and‑drop workflows, high‑performance pipelines, AI engine development, and experience with both massive (80B‑100B) and compact (CPMBee) models. He noted performance optimization on consumer‑grade GPUs, the critical role of prompts, balancing cost and performance, and clarified that RAG primarily improves relevance rather than truthfulness.

The discussion concluded with a summary of the Knowledge Map 3.0, which contains 2,500 knowledge points across 26 domains, contributed by 47 experts, and is offered as a downloadable resource for the new year.