Building a Private Knowledge Base and Large‑Model Platform for Enterprise AI Assistants

Project Background : Following the release of GPT‑3.5 in late February 2023, the information‑system team explored how to harness large AI models to solve internal problems, improve efficiency, and build a system‑level customer‑service platform for repetitive communications.

Solution Overview : The team adopted a private knowledge‑base approach, embedding enterprise documents, policies, and Q&A into vector space, retrieving the most similar entries via cosine similarity, and combining them with role‑prompt instructions before sending the prompt to the LLM.

Private Knowledge Base Construction : Corpus preparation emphasizes semantic understanding, length control, and redundancy to improve similarity scores; embedding vectors are stored locally, and retrieval uses cosine similarity with configurable recall thresholds.

Prompt Engineering : Prompt design includes role definition, prohibited actions, direct answers, contextual knowledge, conversation history, and parameter settings (temperature, max tokens, penalties) to guide the LLM toward accurate, safe responses.

Plugin Extension : To handle real‑time queries (e.g., weather, internal ticket status), the platform supports a plugin mode where the LLM decides whether to call an external API, receives JSON‑formatted results, and then generates a natural‑language answer.

Technical Line Implementation : The backend provides native API integration with multiple LLM providers (ChatGPT, Wenxin, Spark, Tongyi); the data team manages the private knowledge base, plugin registration, and prompt configuration; the front‑end team builds a unified UI with a floating chat button, iframe container, and secure session handling.

Security Measures : Token validation, SSO integration, encrypted cookie tickets, and IP blacklist mechanisms protect access; the system uses Redis for session tracking, WebSocket for real‑time communication, and rate‑limiting per business line to prevent abuse.

Stability and Control : WebSocket channels, Redis‑based online‑user management, real‑time offline handling, and configurable rate limits ensure high availability and controllable resource usage.

Results : The platform quickly delivered AI assistants for procurement, employee support, supplier portals, and other business scenarios on both app and PC, demonstrating the effectiveness of the private knowledge‑base and large‑model middle‑platform.

Conclusion : Private knowledge bases combined with LLM middle‑platforms will become essential for enterprise digital transformation, offering scalable, customizable AI services while maintaining security, stability, and low integration cost.