Why Early Database Management Standards Save You Hundreds of Hours

Guest Introduction

Xiao Peng, manager of Sina's database team, originally responsible for MySQL and now for Redis and HBase, joined Sina in 2010 and has extensive experience in architecture redesign and technical management.

Topic Overview

Database usage management standards are essential yet often overlooked. The discussion covers three aspects: when to introduce standards, how to enforce them, and the strength of enforcement.

When to Introduce Database Usage Standards

In small clusters or early‑stage companies, there is often no dedicated DBA, so developers or part‑time staff maintain the databases without formal standards.

However, can we be lax just because the cluster is small? The author argues against this, citing many cases where early shortcuts caused painful “digging pits” for later teams.

Example: a rapid‑release scenario where an index was added on a slave to boost performance, then later added on the master, leading to data inconsistency due to master‑slave divergence.

Code example:

create table bbb (x int, y int, z int);
insert into bbb values (1,1,1),(3,3,3),(2,2,2),(4,4,4),(5,6,5),(6,5,6);
select y from bbb;

Running the same query before and after adding an index shows different result ordering:

After adding

alter table bbb add index idx_y(y);

and re‑executing

select y from bbb;

, the order changes, demonstrating how non‑standard DML can produce unexpected results.

Conclusion

Establishing standards early avoids costly retrofits later.

How to Implement the Standards

Benefits of Standards

Control: Prevents bizarre SQL and usage patterns. Risk reduction: Enables predefined emergency and optimization plans. DBA interoperability: A DBA familiar with one business can handle another's primary database failures under a common framework.

Implementation approach: Tie standards to processes. For new business onboarding, Sina requires read/write separation, limited CRUD permissions, naming conventions, minimal deployment units, and pre‑sharding for large‑scale services. If a request does not meet these criteria, the DBA can refuse hosting.

In practice, once the standards are defined, an experienced developer can comply within half a day.

Why These Standards Matter

Read/write separation supports horizontal scaling of read traffic. Permission limits prevent destructive actions like drop table . Consistent naming makes table purpose obvious. Deployment architecture ensures high availability and disaster recovery. Pre‑sharding guarantees future scalability.

Without such planning, later bottlenecks can require ten times the effort to resolve, as illustrated by a three‑month, high‑cost core‑system split caused by missing pre‑sharding.

Summary of the Point

Standards must be bound to processes; they either exist or they don’t.

Enforcement Strength

While standards bring many benefits, they can also cause friction, especially when special requirements clash with the rules.

Principle: Once set, standards should not be compromised lightly; designers must anticipate edge cases and maintain consistency, requiring strong leadership.

However, flexibility is sometimes necessary. For example, a business needed

CREATE

permission for a special tool. The team granted a dedicated account with

CREATE

rights limited to specific IPs, preserving overall security while meeting the need.

Design discussions follow the “data speaks” principle: changes are justified by measurable performance improvements.

Key Takeaways

Establish database usage standards early; they benefit future teams. Bind standards to workflows to build habitual compliance. Do not compromise standards lightly; allow limited, controlled exceptions when necessary.

Q&A

Q1: How to prevent mismatched DML between QA and production that leads to unusable indexes?

A1: Use a synchronization system to keep QA schemas aligned with production, enforce SQL review, and capture problematic queries via slow‑query monitoring for DBA optimization.

Q2: Ideal DBA‑to‑developer staffing ratio?

A2: Roughly 1 DBA per 50 developers, but by designating a database liaison per business unit, the effective ratio improves to about 1:5.

Q3: How do you handle permission management?

A3: A custom tool issues separate read and write accounts with randomly generated passwords, replacing a single shared password after a security incident.