Improving International Hotel After‑Sales Service: Metrics, Optimization Strategies, and Risk Prediction with LightGBM

3. After‑sales Experience

3.1 After‑sales Service

The after‑sales experience covers the period from order completion to successful check‑in and check‑out, including confirming orders, handling cancellations, and addressing on‑site issues.

1. After‑sales stages

Four main stages are identified: confirming (order verification by agents, which may take 1 minute to 12 hours), confirmed (order accepted, awaiting check‑in), check‑in (guest arrival and room allocation), and check‑out (post‑stay evaluation and refunds).

2. After‑sales Customer Service

Typical user requests include resending check‑in vouchers, confirming room details, and complaints about missing reservations or rooms.

3.2 Metric Definitions

Two primary metrics are defined to evaluate after‑sales performance:

1. Defect Rate – the weighted ratio of pre‑confirmation rejections, post‑confirmation overturns, and check‑in issues to total orders, adjusted by expert‑derived coefficients.

2. SPO (Service Per Order) – the total service volume per order, composed of phone‑based manual service and online chat service. SPO directly reflects user experience.

3.3 Metric Improvement

1. Reducing Defect Rate

Focus is placed on the confirming stage, where agent‑level rejections are the largest contributor. Strategies include stricter agent control, traffic reallocation to high‑quality agents, and targeted subsidies for order replacement.

2. Reducing SPO

Product redesign of the order‑detail page and code consolidation reduced response time from >2 s to 80 ms, cutting consultation‑type SPO by 60% and overall SPO by ~40%.

Implementation of a crowdsourcing system streamlined non‑standardized user requests, further lowering SPO.

3.4 Predicting Check‑in Defects

To prioritize high‑risk orders for manual verification, a supervised LightGBM decision‑tree model is built using features such as order basics, statistical indicators, and temporal information.

LightGBM offers faster training, lower memory usage, higher accuracy, and native categorical handling.

3. Model Effectiveness

After oversampling to address class imbalance, the test set achieved 0.93 accuracy, 0.93 micro‑average precision, 0.54 macro‑average precision, 0.93 micro‑average recall, 0.67 macro‑average recall, 0.95 F1, and ROC 0.67. The model enables operators to identify ~9.4 risky orders per day, improving confirmation efficiency.

3.5 Impact of the COVID‑19 Pandemic

The early 2020 pandemic severely affected international hotel bookings, leading to a surge in cancellations and changes. The service team worked overtime to process refunds and maintain user satisfaction.

4. Summary of Experience

Key takeaways include the importance of understanding business logic before development and the need for bold, detail‑oriented, rapid‑iteration approaches.

1. Ctrip Patent: OTA platform method for predicting no‑room at check‑in (https://patentimages.storage.googleapis.com/cf/a0/29/bbc97ae761a5d4/CN107506877A.pdf) 2. Paper: Predicting Hotel Booking Cancellation to Decrease Uncertainty and Increase Revenue (https://www.researchgate.net/publication/310504011_Predicting_Hotel_Booking_Cancellation_to_Decrease_Uncertainty_and_Increase_Revenue)