Detecting LVS Traffic Anomalies with Short‑Term and Long‑Term Ratio Algorithms

Introduction

The post, authored by a member of the ADDOPS team responsible for 360 HULK cloud platform operations automation, proposes an efficient LVS traffic anomaly detection algorithm to help operations colleagues precisely identify abnormal traffic surges or drops.

Data Analysis

Seven days of LVS traffic data reveal two patterns: a periodic trend (shown in the first chart) and a random, non‑periodic trend (shown in the second chart). Recognizing the pattern is essential for selecting the appropriate detection strategy.

Detection Mechanism Research

Because time series can be either periodic or non‑periodic, the detection mechanism must handle both cases. The article details four algorithms:

Short‑Term Ratio (SS) : Compare the current value with the previous seven points; if the count of points exceeding a threshold surpasses a preset limit, flag an anomaly.

Dynamic Threshold : Compute the average, max, and min over a recent window, then use the smaller of (max‑avg) and (avg‑min) as a relaxed threshold to reduce false negatives.

Long‑Term Ratio (LS) : Fit a curve over a longer window using EWMA (exponential weighted moving average); apply the 3‑sigma rule on the EWMA residuals to detect deviations.

Chain and Amplitude (CA) : For periodic data, compare the current value against historical values at the same time of day; use static thresholds or amplitude calculations (Δx/x) to identify spikes or drops.

Algorithm Combination

The four methods are grouped by data type: SS and LS for non‑periodic data, Chain and CA for periodic data. Two usage strategies are suggested:

First determine whether the series is periodic (e.g., via differencing or variance‑based tests). Then apply the corresponding branch of algorithms.

Alternatively, ignore periodicity and apply a majority‑vote approach (“few outliers among many”) to flag anomalies.

Code Samples

Python snippets using pandas illustrate the EWMA calculation and static threshold checks:

expAverage = pd.stats.moments.ewma(data, com=50)
stdDev = pd.stats.moments.ewmstd(data, com=50)
if abs(data.values[-1] - expAverage.values[-1]) > 3 * stdDev.values[-1]:
    print "异常"

if new_value > max(past_14_days) * max_threshold:
    print "突增"
if new_value < min(past_14_days) * min_threshold:
    print "突减"

Conclusion

The article presents a suite of LVS traffic anomaly detection techniques, emphasizing that no single method solves every scenario; practitioners must iteratively refine and combine algorithms based on specific operational contexts.

References

1. https://jiroujuan.wordpress.com/2013/10/09/skyline-anomalous-detect-algorithms/

2. http://chuansong.me/n/2032667

3. http://blog.csdn.net/g2V13ah/article/details/78474370