Log Size Reduction Techniques: Methodology and Case Study

In everyday development, developers often print many INFO‑level logs for debugging, which can cause log files to grow beyond thresholds (e.g., 5 GB) as traffic increases, leading to performance issues in log retrieval, data collection, and disk cleanup.

Methodology for log reduction

1. Print only necessary logs : Convert non‑essential INFO logs to DEBUG, or use a temporary switch (via Tracer or RPC context) to promote DEBUG logs to INFO when needed. Example code:

if (log.isDebugEnable()) {
    log.debug(xxx);
} else if (TracerUtils.openDebug2Info()) {
    log.info("【debug2info】" + xxx);
}

2. Merge log entries : Combine consecutive INFO logs into a single line, reducing redundancy.

INFO [traceId] Service before size=10 after size=4

3. Simplify/abbreviate/compress logs : Print only IDs, create lightweight DTOs for logging, or use abbreviations (e.g., w for write, r for read) to shrink payload.

Case study

A pipeline of many beans inherits from an abstract class that logs item counts before and after each bean execution. Optimizations applied:

Print only post‑execution logs or skip when sizes are equal (code snippet shown).

Merge pre‑ and post‑execution size logs into a single entry.

Replace full result objects with simple DTOs containing only key fields (e.g., id, title).

if (sizeBefore != sizeAfter) {
    log.info("service:{}, 前size:{}, 后size:{}", getName(), sizeBefore, sizeAfter);
}
log.info("resultId:{}", result.getId());
log.info("result:{}", toSimpleLog(result));

After applying these techniques, daily log volume dropped from roughly 5 GB to less than 1 GB, with about 80% of the original logs being size‑related entries that were now eliminated or merged.

Conclusion

Effective log slimming requires balancing the need for diagnostic information with storage and performance constraints; strategies include deleting unnecessary logs, merging entries, simplifying payloads, and optionally using a debug‑to‑info switch or tools like Arthas for on‑the‑fly adjustments.