Deploying Domain Models with Open-Source LLMs: Lessons from SECon 2024

Market Context and Model Selection

The domestic large‑model market is shifting toward three trends: rapid improvement of open‑source capabilities, clear separation of ToB and ToC markets, and enterprise focus on deepening core‑business applications.

Choosing an open‑source base model requires balancing implicit abilities (low‑level architectural strengths) and explicit abilities (fine‑tuning potential). The article presents the “ice‑berg theory” that distinguishes these two capability layers and argues that the optimal base must provide a strong foundation for both.

Llama 3 is identified as the strongest current open‑source foundation model, offering leading explicit performance and solid implicit capacity. Its remaining gaps are Chinese language handling, professional‑domain tasks, and online service latency.

Domain Model Construction Paradigm

The construction workflow is organized around three pillars: data , algorithm , and engineering .

Data Pipeline

A four‑stage pipeline—rule filtering → model filtering → deduplication → quality filtering—extracts 32 % of raw Chinese text , producing a 15 TB high‑quality corpus . Dedicated quality‑model checks raise data quality by 48 % . Multiple discriminators (text quality, knowledge relevance, structural consistency) enforce strict standards. Content‑safety filters reduce malicious or sensitive material to <1 % of the corpus.

Algorithmic Enhancements

Three algorithmic stages are applied:

Incremental pre‑training on the curated corpus.

Instruction fine‑tuning, split into mixed‑length pre‑training and standard instruction tuning.

Reinforcement learning for alignment.

The team introduces a mixed‑length bucket training strategy that reduces truncation and improves training throughput by 14.6 % . By preserving long sequences, the model supports up to 100 k token context windows with minimal data overhead.

A self‑generated QA pipeline ( Self‑QA ) converts massive unsupervised text into high‑quality instruction data, enabling cost‑effective fine‑tuning.

Engineering Enhancements

Model efficiency and stability are ensured through:

Quantization (int4/int8) for reduced memory footprint.

Inference acceleration techniques.

Architecture optimizations tailored to the target domain.

Scenario Enhancements

Domain awareness is further boosted by integrating:

Agents for autonomous tool use.

Prompt engineering for task‑specific guidance.

Retrieval‑augmented generation (RAG) to inject external knowledge.

Comprehensive Evaluation Framework

A full‑stack evaluation system conducts:

Horizontal comparisons across different model families.

Vertical assessments of the same model at successive development checkpoints.

Checkpoints automatically trigger evaluation pipelines; results are complemented by manual review and domain‑specific scenario testing, forming a closed‑loop for continuous improvement.

Open‑Source XuanYuan Series

The team released the XuanYuan series (6 B, 13 B, 70 B parameters). XuanYuan‑70B‑V2 outperforms GPT‑4 on the FinanceIQ benchmark and attains top ranks on MMLU, CEVAL, CMMLU, GSM8K, and HumanEval, demonstrating expert‑level financial knowledge and strong general capabilities.

Repository: https://github.com/Duxiaoman-DI/XuanYuan

Key Takeaways

Domain‑specific large models can accelerate enterprise digital transformation across front‑end, middle‑office, and back‑end workflows. Continued advances in base‑model capability, algorithmic and engineering enhancements, and scenario integration are positioned to drive the next phase of AI adoption.