Large Language Model Innovations for the Financial Industry: From General to Finance‑Specific Models, Training Techniques, Evaluation Methods, and Real‑World Applications

The emergence of large language models (LLMs) offers unprecedented capabilities for the data‑driven, knowledge‑intensive, and complex‑process financial industry, prompting a need for finance‑specific models that can handle domain terminology, accuracy requirements, and cost constraints.

General LLMs face three major challenges in finance: specialized knowledge gaps (e.g., KS, MOB, COB terms), hallucination and accuracy issues that conflict with strict financial data standards, and high inference and training costs. Building a dedicated financial LLM therefore becomes essential.

Training a 70B‑parameter model on 2 T tokens would require roughly 187 days on 48 A100 GPUs, and inference would need two A100 cards, making generic models prohibitively expensive for many enterprises. By focusing on domain data, a smaller model can achieve comparable performance, as demonstrated by the XuanYuan series released by Du Xiaoman: a trillion‑parameter Chinese financial model (May 2023), XuanYuan‑70B (Sept 2023), and XuanYuan‑70B‑Chat plus quantized versions (Nov 2023).

Training innovations include extensive data preparation (10 TB of general text plus 1 TB of financial data), incremental pre‑training with a 39K token vocabulary (7 K new Chinese characters), a two‑stage pre‑training schedule (40 B tokens then 300 B tokens), and balanced multilingual token ratios that gradually increase financial data proportion while preserving English capabilities.

Instruction fine‑tuning employs a 4:1 mix of general and finance‑specific SFT data, using a two‑stage approach to maintain generalization while enhancing financial Q&A. Value alignment is achieved via reinforcement learning from human feedback (pairwise ranking) and PPO optimization, focusing on safety, usefulness, and stability.

Engineering optimizations address memory and compute bottlenecks: zero‑redundancy optimizer (Zero) reduces memory by 87 % and triples batch size, flash‑attention and custom kernels boost throughput, and high‑speed networking enables linear scaling up to 640 GPUs with >94 % efficiency.

Evaluation innovations replace reliance on public leaderboards with a comprehensive internal framework that conducts horizontal comparisons across models and vertical tracking of a single model’s progress through pre‑training, fine‑tuning, and reinforcement stages, using both automated metrics and extensive human assessments.

Application breakthroughs span five areas: (1) marketing – real‑time, personalized content generation; (2) service – AI‑augmented customer support achieving ~25 % efficiency gains; (3) operations – unified data‑driven decision making; (4) office – AI assistants that accelerate knowledge acquisition for new employees; and (5) risk control – integrating LLM reasoning with traditional decision‑based risk models for proactive, real‑time risk management.

The article concludes that the iterative path of financial LLMs is driven by a dual loop of model training and evaluation coupled with continuous application feedback, encouraging ongoing learning, curiosity, and collaboration across the industry.