Design of a Real-Time Complex Event Processing System for Xianyu

This article describes the design of a real‑time complex event processing (CEP) system used by the Xianyu platform to handle various business scenarios such as activity coupons, price‑drop notifications, rental recommendations, and promotional offers.

Business Scenarios

场景1：在闲鱼的一次活动中，用户进入活动会场后，浏览了几个不同的宝贝，就会奖励一个包邮券。

场景2：用户关注的用户宝贝降价了，实时告知用户该降价信息。

场景3：在用户搜索租房后，并浏览N个租房信息，则为其推送一套合适的房源。

场景4：双十一会场活动，用户进入会场，点击商品详情，对其发送优惠。

The repeated, case‑by‑case implementation of these scenarios leads to high development cost and low reuse. The solution is to abstract the common pattern: "a series of user actions that satisfy complex rule conditions trigger real‑time rewards". This defines the problem as a real‑time, rule‑driven event processing system.

System Design Goals

1. 技术与业务分离，构建技术组件和能力，组合后实现业务需求；

2. 事件的数据格式需要结构化和标准化，支持扩展；

3. 规则的表达定义类似SQL的声明式DSL，贴合业务领域；

4. 客户端和服务端有各自的行动触发能力，支持扩展开发；客户端支持服务端驱动；

5. 触发和计算分离，计算模式插件化；

The architecture consists of three core modules:

Log Collection Module : An interceptor normalizes logs from heterogeneous front‑end applications (Java, Dart, FaaS) and writes them to SLS. A Blink job with a configurable UDTF cleans and standardizes the logs, then publishes them via RocketMQ and SLS.

EPL Engine Module : A domain‑specific language (DSL) built on top of Blink enables concise rule definition (≈5 lines per task) instead of hundreds of lines of code. The DSL follows industry standards and supports both cloud and edge execution.

Result Delivery Module : Processes EPL results and provides configurable actions such as push, pop‑layer, or open‑page.

Using this pipeline, a new business feature can be launched in half a day with only configuration steps: select log fields, write DSL rules, and configure a delivery action. End‑to‑end latency from data ingestion to user notification is about 10 seconds.

Summary and Outlook : The system decouples heterogeneous services via asynchronous messaging, standardizes data, and offers a DSL for rapid rule development. Future work includes extending DSL expressiveness, adding edge computation, and integrating algorithmic insights for deeper business value.

References: 1. https://arxiv.org/pdf/cs/0612128.pdf (SASE: Complex Event Processing over Streams) 2. http://m.chinaaet.com/article/107902 (RFID‑based CEP research)