Amiya | BestHub

DataFunSummit

Sep 2, 2023 · Big Data

Practical Experience of Bilibili's Big Data Cluster Mixed Deployment Architecture

This article details Bilibili's offline big‑data cluster challenges, the mixed‑deployment architecture that combines offline and online resources, the Amiya service's over‑commit and eviction mechanisms, performance optimizations, monitoring strategies, and future plans to further improve resource utilization and scheduling.

AmiyaBilibiliCluster Scheduling

0 likes · 14 min read

Practical Experience of Bilibili's Big Data Cluster Mixed Deployment Architecture

Bilibili Tech

May 23, 2023 · Big Data

Amiya: Dynamic Overcommit Component for Bilibili Offline Big Data Cluster

Amiya, a self‑developed dynamic over‑commit component for Bilibili’s offline big‑data cluster, inflates reported resources on under‑utilized nodes and adjusts them when load rises, adding roughly 683 TB of memory and 137 k vCores, boosting per‑node memory by 15 % and CPU usage by over 20 % while keeping eviction rates below 3 %.

AmiyaBilibiliResource Overcommit

0 likes · 22 min read

Amiya: Dynamic Overcommit Component for Bilibili Offline Big Data Cluster