Comprehensive Guide to Software Performance Optimization: Indexing, Compression, Caching, Prefetching, Throttling, and Batch Processing

Introduction: Trade‑offs in Performance

Software design is an art of choosing what to take and what to discard; high‑performance systems often require extra resources and may conflict with other quality attributes such as security, scalability, and observability.

The goal is to optimize a system before business bottlenecks appear, using common technical measures that balance time and space.

Part 1 – Six General Techniques (Time ↔ Space)

Indexing

Compression

Caching

Prefetching

Peak‑shaving (削峰填谷)

Batch processing

Indexing

Indexes trade extra storage for faster look‑ups, reducing query complexity from O(n) to O(log n) or O(1). Common data structures include hash tables, binary search trees (e.g., red‑black trees), B‑Tree, B+ Tree, LSM‑Tree, Trie, Skip List, and inverted indexes.

Database‑level advice: define primary keys, create indexes on WHERE/GROUP BY/ORDER BY/JOIN columns, avoid indexing high‑cardinality or frequently updated columns, use composite indexes wisely, and reduce redundant indexes.

Caching

Caching also exchanges space for time and appears at many layers: DNS, OS, CDN, server‑side KV stores, database page cache, CPU caches, and application‑level object pools.

Key challenges: cache invalidation, cache penetration, cache breakdown, and cache avalanche. Mitigations include empty‑value caching, Bloom filters, request coalescing, random TTLs, and careful invalidation strategies.

Compression

Compression trades CPU cycles for reduced data size. Use lossless methods (Gzip/deflate, HTTP/2 HPACK, JS/CSS minification, binary protocols) and understand the limits imposed by information entropy.

Lossy compression is appropriate for media (video, audio, images) where some fidelity loss is acceptable.

Prefetching

Prefetching anticipates future data needs and loads data ahead of time, improving perceived latency in scenarios such as video buffering, HTTP/2 server push, client‑side warm‑up, and server‑side hot‑data pre‑loading.

Peak‑shaving (削峰填谷)

Peak‑shaving smooths traffic spikes by delaying work (e.g., asynchronous queues, rate limiting, back‑off retries) and by scheduling background tasks during off‑peak periods.

Batch Processing

Batching aggregates many small operations into larger ones, reducing per‑operation overhead. Examples include bundling JS/CSS files, bulk Redis MGET/MSET, bulk database inserts (5000‑10000 rows), and message‑queue batch publishing.

Part 2 – Advanced Techniques (Parallelism)

Eight Gates (榨干计算资源)

Focus on reducing system‑call overhead, using zero‑copy I/O, CPU affinity, and DMA to keep CPUs busy with useful work.

Shadow Clone (影分身术) – Horizontal Scaling

Scale stateless services horizontally with load balancers, multiple replicas, auto‑scaling policies, and CDN caching for read‑heavy workloads.

Ultimate Technique (奥义) – Sharding

Partition stateful data across shards, choose appropriate sharding keys, and handle hot‑spot mitigation with multi‑level caches.

Secret Technique (秘术) – Lock‑Free Programming

Eliminate contention using optimistic concurrency (CAS), lock‑free data structures, and pipeline techniques; avoid distributed locks in high‑throughput scenarios like flash sales.

Conclusion

Performance optimization should be driven by measurement (profiling, latency benchmarks) and ROI considerations; avoid premature or over‑optimization, and select the right tools, frameworks, and hardware to achieve the best cost‑benefit balance.