Data Security Construction in Berserker Platform
The article outlines Berserker’s comprehensive data‑security framework—built on the CIA triad and 5A methodology—that unifies authentication, authorization, access control, asset protection, and auditing across Hive, Kafka, ClickHouse and ETL tasks, describes the migration from version 1.0 to 2.0 with a redesigned permission system, workspaces, Casbin performance tweaks, and previews future fine‑grained, lifecycle‑wide security enhancements.
This article introduces the data security construction in Berserker, Bilibili's one-stop data development and governance platform. Berserker is built on common big data ecosystem components and supports various business scenarios including data query, analysis, reporting, integration, development, real-time computing, and governance.
The article focuses on Berserker's data security construction, which provides unified data security management for internal assets like Hive, Kafka, ClickHouse, and ETL tasks, and offers unified functional security control for various data products within the data platform department.
The data security construction follows the CIA triad (Confidentiality, Integrity, Availability) and implements the 5A methodology (Authentication, Authorization, Access Control, Asset Protection, Auditable). The architecture includes identity authentication through company unified authentication and Kerberos, authorization through data security services, access control via Ranger and data security services, asset protection through various measures like download restrictions and data desensitization, and auditing through operation logs and HDFS metadata analysis.
The article details the evolution from version 1.0 to 2.0, highlighting major changes including redesigned permission management system and introduction of workspaces. It discusses challenges faced during the upgrade process, particularly around permission changes and migration. The new permission system simplifies account management, standardizes resource models, and enriches permission types while separating functional and data permissions.
Key issues addressed include Hive table permission migration using HDFS metadata analysis, workspace introduction to better support flexible business needs, and Casbin optimization for performance improvements in permission management. The article also outlines future directions for data security development focusing on lifecycle coverage, fine-grained permission management, sensitive data protection, and risk assessment.
Bilibili Tech
Provides introductions and tutorials on Bilibili-related technologies.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.