Design and Practice of a Risk Control Experiment Platform at Du Xiaoman

Introduction Risk control in internet finance has become mature, with most services moving online, requiring more data‑driven risk management. The Du Xiaoman risk‑control experiment platform provides an environment that spans the entire strategy lifecycle, enabling verification of new or changed policies through both online and offline experiments.

1. Business Background The platform supports two main experiment types: "offline experiments" for validating new or changed policies against test scenarios before deployment, and "small‑traffic online experiments" that evaluate policy changes on a limited traffic slice to observe effects without impacting the full production environment.

2. Platform Architecture Design Practice The system follows a three‑layer architecture: Business Layer (abstracts loan processes into pre‑loan, in‑loan, post‑loan units), Platform Layer (handles variable generation and decision modules), and Data Layer (aggregates internal and external data into topics for analysis and model training). The experiment platform sits within the decision module, linking the full risk‑control decision flow.

3. Architecture Evolution Challenges Challenges include: (a) determining statistical significance of small‑traffic results while minimizing traffic usage; (b) handling variable/feature updates that require re‑computation without contaminating historical samples; (c) mitigating performance overhead of experiment tagging in the online decision path; and (d) reducing latency for large offline sample sets by leveraging offline computation and elastic resources.

4. How to Design a Risk Experiment The design follows five steps: (1) Sample preparation – construct samples via variable back‑tracking; (2) Rule editing – define new rules (e.g., reject credit scores < 500) and replace old ones; (3) Offline experiment – run historical replay experiments using SWAP‑IN/OUT methods; (4) Small‑traffic online experiment – configure T‑test metrics, run until sufficient sample size, and analyze significance; (5) Full rollout – if a significant improvement (e.g., 0.2% lower default rate) is confirmed, promote the experiment to full traffic.

Overall, the platform enables data‑driven, end‑to‑end risk‑control experimentation, balancing statistical rigor, operational efficiency, and system performance.