How to Build a Tool Experience Measurement Model: From Metrics to Behavior Tracking

Tool Experience Measurement Model Construction

CoolHome (酷家乐) explored a systematic experience measurement model for tool‑type products over the past two years, applying it to several tools with positive results. The model addresses the diversity of user types, long and unordered design workflows, and the need for rigorous insight into user intent and behavior.

Tool Product Characteristics

Designers using the tool fall into multiple categories, such as custom designers who start from floor plans, then customize, furnish, and finally render a cabinet design. The workflow includes many design tasks; for example, designing a wardrobe involves placing a frame, adding shelves, accessories, drawers, doors, and then performing a global replacement. User habits lead to varied task sequences.

Thus, tool products exhibit two key traits: (1) diverse user types with vastly different goals, and (2) long, multi‑task workflows where design actions are free‑form and unordered.

Project Positioning

Establish a rigorous system to uncover user intent and behavior, and to monitor experience changes and trends.

Iterate from problem discovery to solution optimization, improving product competitiveness and achieving business goals.

The project is positioned as a tool that assists the business in solving experience problems that affect core business metrics, rather than a top‑down management solution.

Building the Measurement Model

The model breaks down metrics into perception, behavior, and reflection layers, collecting quantitative and qualitative data through usability testing, behavior data, and attitude data. A set of measurement standards and tools were defined to ensure consistent execution.

Multiple Measurement Methods

Attitude measurement: Collects user attitude data; core indicators are NPS (overall experience) and NSS (module‑level satisfaction).

Usability testing: Defines user goals, decomposes tasks, observes users, and follows up with qualitative research on low‑scoring modules.

Behavior measurement: Implements event tracking to analyze user actions, aligning paths with usability tasks for quantitative analysis.

Standardization and Tooling

Standard specifications ensure consistent metric definitions and cross‑team comparability.

Tooling visualizes experience problems, provides dashboards, and supports long‑term monitoring.

Finding Opportunities and Entry Points

Opportunity points are identified by user research and data analysis to uncover experience issues that impact product and business growth. Entry points are defined by dissecting product functions into minimal measurement units based on user paths.

Behavior Measurement: Using User Behavior Data for Experience Evaluation

Indicator Design and Definition

Two primary indicator groups were defined:

Usability indicators: Reach rate, task completion rate, bounce rate.

Efficiency indicators: Task completion time, task completion steps.

Usability Indicators

Reach rate: Percentage of users who enter the defined path start point.

Task completion rate: Core metric measuring whether users can complete the entire path.

Bounce rate: Users who drop off at any touchpoint within the path.

Efficiency Indicators

Task completion time: Time from entering the path to finishing it.

Task completion steps: Number of effective clicks required to finish the path.

Analysis Dimensions and Use Cases

Full Cross‑Analysis Dimensions

Metrics can be cross‑analyzed by similar paths, user types, and time (pre‑ vs post‑optimization).

Scenario 1: Multiple Paths – Single Metric

Compare task completion rates of two similar paths (e.g., searching for a model vs browsing categories).

Compare dependent paths (e.g., selecting a model, placing it, adjusting parameters) to see which path influences downstream usage.

Scenario 2: Single Path – Multiple Metrics

Analyze reach, completion, and bounce rates for a single path.

Further examine efficiency (time vs steps) to identify dominant pain points.

Scenario 3: Multiple Paths – Multiple Metrics

Combine reach and completion rates to compare different entry points.

Pair completion rate with bounce rate to locate problematic touchpoints.

Contrast efficiency metrics across paths to assess time vs step impact.

Behavior Measurement Tool Construction

Challenges and Choices

High tracking cost: Frequent canvas interactions generate massive event volumes.

Imprecise canvas tracking: 3D scene transformations make object identification difficult.

Low efficiency: Manual event definition required for each interaction.

Inconsistent tracking structures: Historical event schemas differ across business lines, hindering cross‑product analysis.

From Full Tracking to Core‑Path Tracking

Because full tracking cannot capture all user actions reliably, the team manually decomposes core paths and their sub‑paths, enabling custom path composition and metric modeling.

Metric Modeling and Toolization

Customizable paths composed of multiple touchpoints.

Metric models automatically fetch data for defined paths.

Support for user segmentation, path saving, and data export for secondary analysis.

Business Practice

Monitor user experience during daily development to detect regressions.

Assist experience designers in decision‑making through path analysis dashboards.

Improve new‑user retention by linking low‑completion paths to weekly retention metrics.

Conclusion

Data‑driven evaluation is essential for design reviews involving cross‑functional stakeholders. By establishing a rigorous measurement system, linking experience metrics to business goals, and continuously iterating, tool products can demonstrate direct value to user growth and commercial outcomes.