DaTaobao Tech
Dec 27, 2023 · Artificial Intelligence
Deploying a Private LLM Knowledge Base on a MacBook
The guide walks through installing and quantizing the open‑source ChatGLM3‑6B model and the m3e‑base embedder on a MacBook, wrapping them with a FastAPI OpenAI‑compatible service, routing requests through a One‑API gateway, storing metadata in MongoDB and vectors in PostgreSQL pgvector, deploying FastGPT for RAG, ingesting data, and demonstrating 5‑7 second response times, while outlining future improvements.
ChatGLM3DeploymentLLM
0 likes · 23 min read