Bilibili Tech
Sep 6, 2022 · Big Data
Lancer: Evolution of Bilibili's Real-Time Streaming Architecture
Lancer, Bilibili’s real‑time streaming backbone, has evolved from a monolithic Flume pipeline to a log‑id‑isolated, Kubernetes‑native architecture where Go edge agents feed synchronous Kafka‑proxied gateways into per‑logid topics processed by dedicated Flink‑SQL jobs, delivering exactly‑once, back‑pressured, highly scalable data ingestion for billions of daily requests.
Big DataData PipelineFlink
0 likes · 29 min read