Building a Robust Data Metric System: Real Cases & Key Challenges

5. Data Metric System Construction – Application Scenarios and Cases

The data metric system is built following the OSM model: Objective (strategic goal) → Strategy (business strategy) → Measurement (metric calculation).

Indicator decomposition follows a hierarchical approach: one North Star metric → 3‑5 primary metrics → 20‑30 secondary metrics.

5.1 Financial Industry Case

5.1.1 Intelligent Risk Control Scenario

Pre‑loan fraud detection identifies risky users using basic information, credit history, device fingerprint, etc., setting indicators such as fraud detection accuracy and multi‑loan index.

During loan monitoring, behavior changes like abnormal transaction time, location, and amount are tracked with indicators such as recent multi‑loan index, abnormal transaction detection rate, and debt‑to‑income ratio.

Post‑loan management focuses on overdue users, monitoring overdue rate, loss‑contact rate, and optimizing collection strategies with metrics like legal success rate and recovery rate.

5.1.2 Application Effectiveness

Objective: Keep credit‑card annual bad‑debt rate under 1.2%.

Pre‑loan fraud interception

Mid‑loan abnormal transaction monitoring

Post‑loan collection strategy optimization

Measurement (hierarchical metric system):

5.2 E‑commerce Case

5.2.1 "People, Goods, Place" Scenario

The e‑commerce metric system revolves around three dimensions:

People : Covers the full user lifecycle—acquisition, registration, onboarding, growth, retention, repurchase, and churn warning. Indicators include potential user click‑through rate, registration conversion, first‑purchase cycle, repeat purchase rate, and churn rate.

Goods : Tracks product lifecycle from listing to after‑sale. Indicators include product exposure, click‑through rate, add‑to‑cart rate, inventory turnover, positive review rate, and return rate.

Place : Encompasses platform pages, search, category channels, and marketing activity pages. Indicators include page views (PV), unique visitors (UV), dwell time, bounce rate, conversion rate, and activity‑specific metrics such as coupon redemption and sales contribution.

5.2.2 Application Effectiveness

Objective: Increase GMV during major promotions to 5 billion CNY.

Precise user operation (People)

Hot‑product traffic support (Goods)

Scenario experience optimization (Place)

Measurement (hierarchical metric system):

6. Challenges and Countermeasures in Data Metric System Construction

6.1 Data Quality Issues

Key challenges include data accuracy, completeness, and consistency. Problems arise from source contamination (e.g., missing event tracking, inconsistent definitions), transmission distortion (e.g., delayed sync, queue backlog), and processing anomalies (e.g., task failures).

6.1.2 Countermeasures

Technical Governance : Implement multi‑layer data validation, including:

Collection validation: real‑time checks in SDK for required fields and format.

Transmission monitoring: alert if latency exceeds 5 minutes.

Processing monitoring: track task status and perform full‑lineage field‑level diff analysis.

Management Mechanism : Introduce data‑quality KPIs into OKRs, such as department‑level data accuracy targets.

6.2 Organizational Collaboration Barriers

Cross‑departmental conflicts (e.g., differing metric definitions, resource competition) and unclear responsibilities hinder metric system effectiveness.

6.2.2 Countermeasures

Build an enterprise‑wide data governance framework that cultivates a data‑driven culture through training, success‑case sharing, and clear ownership.

6.3 Sustainability of the Metric System

Issues include metric inflation (hundreds of new metrics annually, many unused), technical debt (stale calculation logic), and maintenance overhead (large effort to retire obsolete metrics).

6.3.2 Countermeasures

Metric Management Mechanism: Define a health index = usage frequency × impact weight × update timeliness.

Lifecycle Management: Adopt a four‑stage state machine (Observation → Trial → Formal → Retire).

Agile Iteration: Quarterly eliminate metrics with utilization < 5 % (e.g., a bank reduced redundant metrics by 30 %).

Intelligent Operations: Automate metric rule validation with scripts for continuous monitoring and alerts.

6.4 Measuring Metric Value

Companies struggle to quantify the ROI of metric systems; many dashboards receive little usage despite heavy investment.

6.4.2 Countermeasures

Establish a metric value assessment framework covering:

Business Value : Use attribution analysis to link metric improvements (e.g., 1 % conversion lift) to revenue gains.

Technical Value : Quantify resource savings from faster queries and unified data processing.

- End -