Redesign of Bilibili Edge CDN Architecture: From Legacy to Cloud‑Native Load‑Balancing

The article presents a comprehensive redesign of Bilibili's downstream CDN architecture, addressing the limitations of the legacy system and introducing a cloud‑native solution.

Background : The old CDN architecture tightly coupled edge CDN nodes with a central scheduling service. Traffic was first balanced by the scheduler, then converged by a back‑origin component. As traffic grew, this model caused frequent incidents, slow fault recovery (up to 20 minutes), low resource utilization, and complex user‑experience improvements.

Requirements identified include:

Gateway components must support both Layer‑7 and Layer‑4 load balancing.

A cluster‑wide health‑check mechanism to detect overloads and faulty components.

All routing policies should be centralized in a control component to minimize changes to existing services.

New Architecture Design : The redesigned system introduces four main components:

Gateway component – handles user protocols (H1, H2, H3), authentication, and traffic convergence.

Cache component – focuses on disk I/O, file systems, and eviction policies.

Back‑origin component – optimizes back‑origin protocols and speeds.

Control component – manages intra‑cluster traffic routing, hot‑flow dispersion, and faulty component eviction.

Gateway Load‑Balancing :

Layer‑4 load balancing (implemented by another team) solves load imbalance caused by the scheduler and quickly diverts traffic from faulty Layer‑7 nodes.

Layer‑7 load balancing isolates problematic cache or back‑origin components, rerouting traffic to healthy nodes while preserving user playback experience.

Control Component : Acts as a multi‑node service with consistent query interfaces, orchestrating other components and ensuring data consistency.

Load‑Balancing Strategies :

Live streaming: Traffic is divided into rectangular slots; when a slot exceeds a threshold, excess traffic is dispersed to other components.

VOD: Files are split into chunks and distributed using consistent hashing.

NGINX Slice Module Refactor :

The original http_slice_module caused read amplification and sequential sub‑request delays. The new design introduces a prefetch window that allows two concurrent sub‑requests, reducing memory pressure and latency.

Code snippet illustrating the refactor:

typedef struct {
    ...
    ngx_http_request_t*  prefetch[2];
} ngx_http_slice_ctx_t;

ngx_http_slice_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
{
    ...
    if (ctx == NULL || r != r->main) {
        // Ensure previous prefetch sub‑request completed before next slice
        if (ctx && ctx->active) {
            rc = ngx_http_slice_prefetch(r->main, ctx);
            if (rc != NGX_OK) {
                return rc;
            }
        }
        return ngx_http_next_body_filter(r, in);
    }
    ...
    rc = ngx_http_next_body_filter(r, in);
    if (rc == NGX_ERROR || !ctx->last) {
        return rc;
    }
    if (ctx->start >= ctx->end) {
        ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
        ngx_http_send_special(r, NGX_HTTP_LAST);
        return rc;
    }
    if (r->buffered) {
        return rc;
    }
    if (ctx->active) {
        // slice prefetch
        rc = ngx_http_slice_prefetch(r->main, ctx);
    }
    return rc;
}

ngx_http_slice_prefetch(ngx_http_request_t *r, ngx_http_slice_ctx_t *ctx)
{
    // control prefetch window
    if (ctx->prefetch[1]) {
        if (!ctx->prefetch[0]->done) {
            return NGX_OK;
        }
        ctx->prefetch[0] = ctx->prefetch[1];
        ctx->prefetch[1] = NULL;
    }
    if (ctx->start >= ctx->end) {
        return NGX_OK;
    }
    ...
    if (ngx_http_subrequest(r, &r->uri, &r->args, &ctx->prefetch[1], ps, NGX_HTTP_SUBREQUEST_CLONE) != NGX_OK) {
        return NGX_ERROR;
    }
    ngx_http_set_ctx(ctx->prefetch[1], ctx, ngx_http_slice_filter_module);
    ctx->active = 0;
    ...
    if (!ctx->prefetch[0]) {
        ctx->prefetch[0] = ctx->prefetch[1];
        ctx->prefetch[1] = NULL;
    }
    ngx_http_slice_loc_conf_t *slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
    off_t cur_end = ctx->start + get_slice_size(ctx->start, slcf);
    if (slcf->shrink_to_fit) {
        cur_end = ngx_min(ctx->end, cur_end);
    }
    gen_range(ctx->start, cur_end - 1, &ctx->range);
    ...
}

The refactor reduced request processing time by about 40 % compared to the original implementation.

Business Impact :

Mixed live/VOD workloads achieve better resource utilization: VOD benefits from DMA‑offloaded disk I/O, while live streams gain extra CPU for protocol handling.

Pre‑heating becomes automatic; the control component detects hot flows and triggers targeted cache warm‑up.

Storage resources are maximized by allowing gateways to access any cache node, enabling dynamic re‑allocation based on disk‑IO performance.

Monitoring data shows a significant reduction in SLO alarms and more stable service after the new architecture is gradually rolled out.