BestHub
Sign in
Tagged articles
190 articles
Page 1 of 2
ITPUB
ITPUB
May 28, 2026 · Operations

10 Essential Shell Scripts for Efficient Log Management

This article provides ten ready‑to‑use Bash scripts that cover real‑time log monitoring, daily rotation, error aggregation across servers, timestamp conversion, IP address statistics, request‑type analysis, response‑time bucketing, file‑diff comparison, user‑activity tracking, and log compression with optional remote backup, helping operators automate and streamline log handling tasks.

BashLog ManagementShell
0 likes · 10 min read
10 Essential Shell Scripts for Efficient Log Management
ITPUB
ITPUB
May 25, 2026 · Operations

Why Manually Pulling Server Logs Is Inefficient: Comparing ELK, EFK, and PLG Stacks

The article compares popular log‑collection stacks—ELK/Elastic Stack, EFK with Fluent Bit, and the PLG solution (Promtail + Loki + Grafana)—detailing their components, deployment scenarios, and trade‑offs such as indexing strategy, storage options, and integration with Kubernetes for observability.

EFKELKGrafana
0 likes · 5 min read
Why Manually Pulling Server Logs Is Inefficient: Comparing ELK, EFK, and PLG Stacks
MaGe Linux Operations
MaGe Linux Operations
Apr 29, 2026 · Operations

Step‑by‑Step Investigation of a High‑Load Production Server

During a mid‑year promotion an e‑commerce platform experienced a sudden spike in load average and response latency; the article walks through a systematic, command‑driven investigation that identifies an I/O bottleneck caused by mis‑configured log rotation and excessive debug logging, and presents immediate and long‑term remediation steps.

I/OLinuxLog Management
0 likes · 16 min read
Step‑by‑Step Investigation of a High‑Load Production Server
Java Tech Enthusiast
Java Tech Enthusiast
Mar 27, 2026 · Operations

How to Quickly Diagnose and Resolve Disk Space Exhaustion in Production

This guide walks through a step‑by‑step process for identifying the partitions and files that fill a disk, applying temporary fixes to bring usage below critical levels, and implementing long‑term measures to prevent future disk‑full incidents in production environments.

LinuxLog ManagementSystem Administration
0 likes · 9 min read
How to Quickly Diagnose and Resolve Disk Space Exhaustion in Production
Efficient Ops
Efficient Ops
Mar 25, 2026 · Operations

Why VictoriaLogs Is the Fastest, Most Efficient Log Database for Modern Infrastructures

VictoriaLogs, an open‑source log management database from the VictoriaMetrics ecosystem, offers ultra‑low resource consumption, multi‑protocol ingestion, powerful LogsQL queries, and flexible deployment options—including single‑node, Docker, and clustered setups—making it a compelling alternative to Elasticsearch and Loki.

DockerLinuxLog Management
0 likes · 6 min read
Why VictoriaLogs Is the Fastest, Most Efficient Log Database for Modern Infrastructures
Ops Community
Ops Community
Mar 14, 2026 · Operations

How to Diagnose, Clean, and Prevent Docker Log Disk Exhaustion

This guide walks you through identifying which Docker containers are consuming disk space, safely truncating oversized log files, configuring log drivers and rotation policies, setting up centralized logging, and automating cleanup to avoid future disk‑full incidents in production environments.

ContainerDevOpsDocker
0 likes · 33 min read
How to Diagnose, Clean, and Prevent Docker Log Disk Exhaustion
Raymond Ops
Raymond Ops
Feb 5, 2026 · Operations

Midnight Log Nightmare: Choosing the Right Linux Log Management Tool

A 3 AM production outage reveals how massive, unindexed log files can cripple incident response, prompting a detailed comparison of traditional text tools, log rotation, ELK Stack, and Grafana Loki, along with practical tips, common pitfalls, and future trends in log management.

ELKGrafana LokiLog Management
0 likes · 13 min read
Midnight Log Nightmare: Choosing the Right Linux Log Management Tool
Xiao Liu Lab
Xiao Liu Lab
Jan 7, 2026 · Operations

Zero‑IO Techniques to Safely Clear Massive Log Files on Linux

This article explains why deleting huge log files can crash a server, compares the low‑IO echo and truncate commands for safely emptying logs, provides practical examples, parameter tips, additional methods, and best‑practice recommendations for production environments.

EchoIO optimizationLinux
0 likes · 10 min read
Zero‑IO Techniques to Safely Clear Massive Log Files on Linux
Efficient Ops
Efficient Ops
Jan 4, 2026 · Databases

Why VictoriaLogs Is the Efficient Open‑Source Log Database You Need

VictoriaLogs is a fast, resource‑light, open‑source log database from the VictoriaMetrics ecosystem that offers low RAM and disk usage, multi‑protocol ingestion, powerful LogsQL queries, and flexible deployment options ranging from single‑node binaries to Docker and clustered setups.

ClusterDockerInstallation
0 likes · 6 min read
Why VictoriaLogs Is the Efficient Open‑Source Log Database You Need
Alibaba Cloud Observability
Alibaba Cloud Observability
Dec 29, 2025 · Cloud Native

How Alibaba Cloud Log Service Supercharges Dify’s Scaling and Cuts DB Costs

This article examines Dify’s production‑scale bottlenecks caused by heavy PostgreSQL logging, explains why a cloud‑native log service (SLS) better matches the append‑only, high‑throughput nature of workflow logs, and provides a step‑by‑step migration guide that dramatically reduces database pressure, storage cost, and unlocks advanced analytics.

Alibaba Cloud Log ServiceCloud NativeDify
0 likes · 17 min read
How Alibaba Cloud Log Service Supercharges Dify’s Scaling and Cuts DB Costs
Code Ape Tech Column
Code Ape Tech Column
Dec 19, 2025 · Backend Development

Boost SpringBoot Log Management: Step‑by‑Step Integration with Hera

This article explains why traditional SpringBoot logging falls short, introduces the Hera log platform’s three core benefits, outlines a layered integration architecture, and provides a detailed five‑step guide—including Maven dependencies, YAML configuration, custom field providers, log output, traceability, and console usage—plus performance, high‑availability, security tips and common pitfalls.

Distributed SystemsHeraLog Management
0 likes · 14 min read
Boost SpringBoot Log Management: Step‑by‑Step Integration with Hera
Raymond Ops
Raymond Ops
Dec 8, 2025 · Operations

Mastering EFK: Complete Guide to Building a Scalable Log Management Solution

This comprehensive guide walks you through building a scalable EFK log management solution, covering architecture components, high‑availability design, environment preparation, detailed Elasticsearch, Fluentd and Kibana deployment steps, index optimization, monitoring, alerting, security hardening, troubleshooting and best‑practice recommendations for modern cloud‑native operations.

EFKElasticsearchFluentd
0 likes · 19 min read
Mastering EFK: Complete Guide to Building a Scalable Log Management Solution
Su San Talks Tech
Su San Talks Tech
Nov 18, 2025 · Backend Development

Boost SpringBoot Production Deployments with a Visual Service Manager

This guide presents a visual, configuration‑driven service manager for SpringBoot applications that streamlines start/stop operations, provides real‑time status and resource monitoring, offers intelligent log handling, supports batch actions, and includes an automated deployment and rollback workflow to improve operational efficiency and reliability.

Deployment AutomationLog ManagementMonitoring
0 likes · 23 min read
Boost SpringBoot Production Deployments with a Visual Service Manager
Raymond Ops
Raymond Ops
Nov 8, 2025 · Operations

Mastering journalctl: Persistent Systemd Log Management and Cleanup

This guide explains how to use systemd's journalctl for reading, configuring persistent storage, limiting log size, setting up automatic cleanup, and verifying integrity of Linux logs, providing practical commands and configuration snippets for reliable log management.

LinuxLog Managementjournalctl
0 likes · 9 min read
Mastering journalctl: Persistent Systemd Log Management and Cleanup
Architect
Architect
Nov 6, 2025 · Operations

Why Most Teams Should Choose Loki Over ELK for Log Management – A Cost‑Effective Guide

This comprehensive guide compares ELK, EFK, and Loki log‑management solutions, analyzing their architecture, performance, cost, and use‑case suitability, and provides a decision framework, real‑world case studies, migration strategies, and optimization tips to help teams select the most efficient logging stack for their needs.

ELKLog ManagementLoki
0 likes · 36 min read
Why Most Teams Should Choose Loki Over ELK for Log Management – A Cost‑Effective Guide
Ops Community
Ops Community
Nov 2, 2025 · Operations

Zero‑Downtime Nginx Log Rotation: Full Logrotate Automation & Compression Guide

Learn how to achieve zero‑downtime Nginx log rotation using Logrotate, covering prerequisites, environment matrix, step‑by‑step configuration, compression strategies, advanced slicing, automated cleanup, monitoring, performance tuning, remote archiving, security hardening, and troubleshooting to ensure continuous logging without service interruption.

LinuxLog ManagementZero Downtime
0 likes · 30 min read
Zero‑Downtime Nginx Log Rotation: Full Logrotate Automation & Compression Guide
Raymond Ops
Raymond Ops
Oct 21, 2025 · Big Data

Deep Dive into Kafka Architecture: Topics, Partitions, and Reliable Data Pipelines

This article explains Kafka’s core concepts—including topics, partitions, log segmentation, indexing, and acknowledgment mechanisms—then provides a step‑by‑step guide to deploy a Zookeeper‑Kafka cluster integrated with Filebeat, Logstash, and the ELK stack for reliable log collection and analysis.

Big DataELKFilebeat
0 likes · 11 min read
Deep Dive into Kafka Architecture: Topics, Partitions, and Reliable Data Pipelines
MaGe Linux Operations
MaGe Linux Operations
Oct 14, 2025 · Cloud Native

How Loki + S3 Cuts Log Storage Costs by Up to 90% at PB Scale

This article explains how the cloud‑native Loki logging system combined with S3 object storage can reduce PB‑level log storage expenses by 80‑90%, while simplifying operations, improving query performance, and meeting compliance requirements through detailed architecture, configuration, deployment, and real‑world case studies.

Log ManagementLokiObservability
0 likes · 23 min read
How Loki + S3 Cuts Log Storage Costs by Up to 90% at PB Scale
MaGe Linux Operations
MaGe Linux Operations
Oct 12, 2025 · Operations

How to Balance Loki Tag Design and Chunk Compression to Tame Log Floods

Learn how to design low‑cardinality Loki tags, fine‑tune Chunk compression settings, and implement best‑practice configurations, pipelines, and monitoring to prevent memory overload, improve query performance, and efficiently manage massive log volumes in cloud‑native environments.

Log ManagementLokiObservability
0 likes · 38 min read
How to Balance Loki Tag Design and Chunk Compression to Tame Log Floods
MaGe Linux Operations
MaGe Linux Operations
Oct 5, 2025 · Operations

ELK vs EFK vs Loki: Which Log Solution Saves Money and Boosts Performance?

This in‑depth technical guide compares ELK, EFK, and Loki across cost, performance, deployment complexity, feature completeness, and suitability for small‑to‑large teams, providing real‑world case studies, decision trees, migration steps, and cost‑optimization tips to help you choose the most efficient logging stack for your organization.

EFKELKLog Management
0 likes · 39 min read
ELK vs EFK vs Loki: Which Log Solution Saves Money and Boosts Performance?
Alibaba Cloud Observability
Alibaba Cloud Observability
Sep 29, 2025 · Cloud Native

How Alibaba Cloud SLS Soft Delete Enables Instant, Low‑Cost Data Cleanup

This article explains Alibaba Cloud's Log Service (SLS) soft‑delete feature, describing its mark‑and‑filter mechanism, implementation steps, and real‑world scenarios where it replaces costly hard‑delete or ETL solutions with near‑instant, low‑impact data removal for compliance, emergencies, and test‑data contamination.

Alibaba CloudCloud NativeLog Management
0 likes · 9 min read
How Alibaba Cloud SLS Soft Delete Enables Instant, Low‑Cost Data Cleanup
Ops Community
Ops Community
Sep 15, 2025 · Cloud Native

Master Kubernetes Log Collection: From Basics to Advanced EFK & Loki Solutions

This comprehensive guide explains why log management is critical for large Kubernetes clusters, outlines common pain points, presents full‑stack architectures, details EFK and Loki implementations with code samples, and offers performance, security, cost‑optimization, and future‑trend recommendations.

Cloud NativeEFKKubernetes
0 likes · 16 min read
Master Kubernetes Log Collection: From Basics to Advanced EFK & Loki Solutions
Code Mala Tang
Code Mala Tang
Aug 30, 2025 · Backend Development

How to Log API Requests Without Slowing Down Your Server

Effective API logging is essential for debugging and compliance, but naive synchronous logging can block the event loop, exhaust disk I/O, and degrade performance; this guide explains why, and provides ten practical steps—including asynchronous loggers, buffering, offloading, sensitive data masking, and monitoring—to keep your server fast and reliable.

API loggingAsynchronousLog Management
0 likes · 15 min read
How to Log API Requests Without Slowing Down Your Server
Raymond Ops
Raymond Ops
Aug 14, 2025 · Databases

Master MySQL Log Management: Error, General, Binlog & Slow Query Tips

This guide explains MySQL logging types—including error, general, binary, and slow query logs—their default settings, how to enable or modify them, how to view and interpret log contents, and practical commands for backup, recovery, and cleanup.

Database AdministrationLog Managementbinary log
0 likes · 11 min read
Master MySQL Log Management: Error, General, Binlog & Slow Query Tips
MaGe Linux Operations
MaGe Linux Operations
Jul 12, 2025 · Operations

Mastering EFK: The Complete Guide to Building a Scalable Log Management System

This comprehensive guide explains the EFK (Elasticsearch, Fluentd, Kibana) log management stack, covering its components, architecture, deployment steps, log collection strategies, index optimization, monitoring, security hardening, troubleshooting and best‑practice recommendations for building a reliable, scalable logging solution in modern cloud‑native environments.

DockerEFKElasticsearch
0 likes · 17 min read
Mastering EFK: The Complete Guide to Building a Scalable Log Management System
macrozheng
macrozheng
Jul 1, 2025 · Operations

Best Log Management Tools Compared: Filebeat, Graylog, ELK, Loki, Datadog & More

This article provides a comprehensive comparison of popular log management solutions—including Filebeat, Graylog, the Elastic (ELK) stack, Grafana Loki, LogDNA, Datadog, Logstash, Fluentd, and Splunk—detailing their main features, pricing models, advantages, and drawbacks to help you choose the right tool for your needs.

ELK StackLog ManagementObservability
0 likes · 16 min read
Best Log Management Tools Compared: Filebeat, Graylog, ELK, Loki, Datadog & More
Alibaba Cloud Native
Alibaba Cloud Native
Jun 18, 2025 · Operations

Avoid These 6 Log Management Anti‑Patterns to Keep Your Cloud‑Native Systems Reliable

Effective log management is crucial for cloud‑native observability, yet common practices like copy‑truncate rotation, NAS storage, multi‑process writes, file‑hole creation, frequent overwrites, and vim edits can cause data loss or duplicate collection; adopting create‑mode rotation, local disks, append‑only writes, and proper tools mitigates these risks.

Cloud NativeLog ManagementOperations
0 likes · 10 min read
Avoid These 6 Log Management Anti‑Patterns to Keep Your Cloud‑Native Systems Reliable
Raymond Ops
Raymond Ops
Jun 3, 2025 · Operations

Understanding Linux Inodes, File Operations, and Log Management

This article explains Linux inode table structure, how commands like ls, stat, cp, mv, and rm interact with inodes, compares hard and soft links, outlines common system log files and their formats, and demonstrates centralizing logs on a rsyslog server.

LinuxLog ManagementSystem Administration
0 likes · 9 min read
Understanding Linux Inodes, File Operations, and Log Management
dbaplus Community
dbaplus Community
Jun 1, 2025 · Backend Development

Why Log4j2 Async Logging Blocks Threads and How to Fix It

This article examines Log4j2 asynchronous logging bottlenecks, explains the underlying Disruptor queue mechanics, identifies root causes of thread blocking, and presents practical strategies—including queue tuning, log classification, bytecode instrumentation, and IDE plugins—to achieve fine‑grained, performant log control.

DisruptorIDE pluginLog Management
0 likes · 12 min read
Why Log4j2 Async Logging Blocks Threads and How to Fix It
Open Source Linux
Open Source Linux
May 26, 2025 · Operations

Master Linux Log Management: rsyslog, systemd‑journal, and Logrotate Explained

Learn how to configure and manage Linux logging services—including rsyslog and systemd‑journal—by understanding common log files, severity levels, log types, server setup, security settings, manual syslog transmission, journal analysis, persistent storage, and log rotation with logrotate to ensure reliable system monitoring.

Log ManagementSystem Administrationlogrotate
0 likes · 15 min read
Master Linux Log Management: rsyslog, systemd‑journal, and Logrotate Explained
Baidu Geek Talk
Baidu Geek Talk
Apr 28, 2025 · Operations

How Baidu’s Log Platform Cuts Billions in Cost with Full‑Lifecycle Event Governance

This article details Baidu's log platform point‑governance practice, explaining why uncontrolled event logging inflates storage and compute costs, and describing a three‑stage solution—manual, semi‑automatic platform, and full‑lifecycle standardization—that uses anomaly detection, automated workflows, and IM bots to achieve massive PV reduction and annual cost savings.

Log ManagementOperationsPlatform Engineering
0 likes · 20 min read
How Baidu’s Log Platform Cuts Billions in Cost with Full‑Lifecycle Event Governance
Aikesheng Open Source Community
Aikesheng Open Source Community
Feb 13, 2025 · Databases

Troubleshooting OceanBase Single‑Node Replica Expansion and Log Disk Size Issues

This article details a step‑by‑step investigation of OceanBase single‑node replica expansion failures, highlighting missing sys‑tenant expansion, deprecated table replica commands, log_disk_size misconfiguration, log‑stream mechanics, and provides concrete SQL and ALTER statements to reproduce and resolve the issue.

Log ManagementOceanBaseTroubleshooting
0 likes · 18 min read
Troubleshooting OceanBase Single‑Node Replica Expansion and Log Disk Size Issues
MaGe Linux Operations
MaGe Linux Operations
Jan 29, 2025 · Databases

Master MySQL Log Management: From Error to Slow Query Logs

This guide explains MySQL’s error, general, binary, and slow query logs, covering default settings, how to enable or modify them, binary log formats and recovery procedures, as well as tools for analyzing slow queries, providing a comprehensive reference for database administrators.

Database AdministrationLog ManagementSQL
0 likes · 11 min read
Master MySQL Log Management: From Error to Slow Query Logs
Architect
Architect
Dec 21, 2024 · Operations

Log Governance and Mining Solution for Distributed Systems

This article presents a comprehensive log governance solution that standardizes, integrates, and optimizes distributed system logs—covering traceability, performance analysis, metric monitoring, and large‑payload handling—to improve observability, reduce resource consumption, and enable effective data‑driven decision making.

Log ManagementTraceabilitylogging
0 likes · 11 min read
Log Governance and Mining Solution for Distributed Systems
Alibaba Cloud Native
Alibaba Cloud Native
Dec 11, 2024 · Information Security

How to Secure Cloud Logs: End‑to‑End Sensitive Data Scanning and Masking on Alibaba Cloud

This guide walks through why enterprises need to scan and protect sensitive log data, explains the regulatory background, and provides a step‑by‑step implementation on Alibaba Cloud using Data Security Center, Logtail, SPL, Ingest Processors, SDKs, StoreView queries and periodic scans to achieve comprehensive data security and governance.

Data SecurityLog Managementcloud-native
0 likes · 18 min read
How to Secure Cloud Logs: End‑to‑End Sensitive Data Scanning and Masking on Alibaba Cloud
Top Architect
Top Architect
Oct 26, 2024 · Operations

Centralized Log Collection with Filebeat and Graylog

This article explains how to use Filebeat as a lightweight log shipper together with Graylog, Elasticsearch, and MongoDB to collect, process, and visualize logs from multiple environments, including detailed configuration examples, deployment scripts, and integration with Docker and Spring Boot.

DockerElasticsearchFilebeat
0 likes · 19 min read
Centralized Log Collection with Filebeat and Graylog
ByteDance Data Platform
ByteDance Data Platform
Oct 9, 2024 · Big Data

Douyin’s E‑commerce Tracking Journey: From Log 1.0 to a Unified Attribution Platform

This article examines Douyin Group’s e‑commerce data‑tracking evolution, detailing the transition from early log‑free collection through Log 2.0’s failed overhaul to the streamlined Log 3.0 framework, and explains the resulting SDK, BTM/BCM management, and attribution platform that solve quality, efficiency, and analysis challenges for data engineers.

Log ManagementSDKdata attribution
0 likes · 19 min read
Douyin’s E‑commerce Tracking Journey: From Log 1.0 to a Unified Attribution Platform
MaGe Linux Operations
MaGe Linux Operations
Sep 22, 2024 · Operations

Master Linux Logging: rsyslog, journald, and logrotate Explained

This guide walks through Linux logging fundamentals, covering rsyslog service architecture, journald configuration, and logrotate management, while showing how to customize log destinations, use selectors and actions, forward logs over the network, and employ systemd timers for automated rotation.

Linux loggingLog ManagementSystemd
0 likes · 23 min read
Master Linux Logging: rsyslog, journald, and logrotate Explained
Software Development Quality
Software Development Quality
Jul 11, 2024 · Information Security

How to Implement Secure and Compliant Log Management Standards

This guide outlines the purpose, scope, principles, and detailed specifications for log management—including file naming, retention periods, content rules, security handling, and monitoring—to ensure reliable issue tracing, data safety, and regulatory compliance across all system development projects.

ComplianceLog ManagementMonitoring
0 likes · 12 min read
How to Implement Secure and Compliant Log Management Standards
DevOps Operations Practice
DevOps Operations Practice
Jul 8, 2024 · Operations

Common Linux Log Files and Useful Commands for Log Analysis

This article introduces common Linux log files, essential command‑line utilities such as cat, less, tail, grep, awk, and sed for viewing logs, and powerful analysis tools like logrotate, journalctl, logwatch, and goaccess, providing practical examples for effective system log management.

Log ManagementSystem Administrationlog analysis
0 likes · 7 min read
Common Linux Log Files and Useful Commands for Log Analysis
Liangxu Linux
Liangxu Linux
May 28, 2024 · Operations

Master Linux Log Management: Crontab Cleanup and Logrotate Best Practices

This guide explains how to effectively manage application logs on Linux by defining log content and archiving rules, handling special cases for services like MySQL and nginx, using crontab for scheduled cleanup, and configuring logrotate with detailed options and examples.

LinuxLog Managementcrontab
0 likes · 11 min read
Master Linux Log Management: Crontab Cleanup and Logrotate Best Practices
Liangxu Linux
Liangxu Linux
May 20, 2024 · Operations

Mastering Linux Log Management with crontab and logrotate

This guide explains how to define log management goals, handle special logging scenarios for services like MySQL and nginx, and implement automated cleanup using crontab and logrotate, including detailed configuration options and practical examples for reliable log rotation and storage control.

LinuxLog ManagementSystem Administration
0 likes · 12 min read
Mastering Linux Log Management with crontab and logrotate
Ctrip Technology
Ctrip Technology
May 17, 2024 · Backend Development

Comprehensive Log Governance and Mining Solution for Distributed Systems

This article presents a comprehensive log governance and mining solution for distributed systems, covering background challenges, usage scenarios, and detailed strategies such as distributed log integration, front‑back end traceability, standardized log management, large‑payload handling, efficient cleaning, and future plans for componentization and sampling.

Distributed SystemsLog ManagementPerformance Optimization
0 likes · 16 min read
Comprehensive Log Governance and Mining Solution for Distributed Systems
Liangxu Linux
Liangxu Linux
Apr 17, 2024 · Operations

Master Linux System Logs: Essential Commands, Files, and Tools

This guide explains how to view and manage Linux system logs using command‑line utilities like journalctl and dmesg, direct file access, and graphical tools, providing practical examples, common log file locations, and tips for filtering and monitoring log output.

LinuxLog Managementdmesg
0 likes · 5 min read
Master Linux System Logs: Essential Commands, Files, and Tools
Zhuanzhuan Tech
Zhuanzhuan Tech
Feb 7, 2024 · Operations

Recovering a ZooKeeper Cluster with Codis: Diagnosis, Testing, and Migration Strategies

This article details a real‑world investigation of a ZooKeeper election‑port failure that prevented adding observer nodes to a Codis cache cluster, outlines systematic connectivity checks, log analysis, and two migration plans, and finally presents step‑by‑step procedures for rolling upgrades, configuration adjustments, and successful cluster restoration.

Cluster RecoveryCodisLog Management
0 likes · 12 min read
Recovering a ZooKeeper Cluster with Codis: Diagnosis, Testing, and Migration Strategies
DaTaobao Tech
DaTaobao Tech
Feb 7, 2024 · Backend Development

Log Management Practices for Reducing Cost and Improving Efficiency

The article presents a log‑governance case study that tackles classification, format, and tool chaos by introducing a three‑layer log hierarchy, simple non‑overlapping classification, fixed‑order CSV formatting, and reverse‑printed stack traces, achieving up to 88 % size reduction while improving cost efficiency and adaptability.

Log ManagementPerformanceStackTrace
0 likes · 15 min read
Log Management Practices for Reducing Cost and Improving Efficiency
dbaplus Community
dbaplus Community
Jan 23, 2024 · Operations

How We Built a Scalable Real‑Time Log Center with ClickHouse and ELK

Facing massive data volumes, the team at Kuaidi100 redesigned their logging platform, moving from a file‑based system to an ELK stack and finally to a ClickHouse‑based architecture, achieving real‑time, scalable, cost‑effective log collection, analysis, and alerting while addressing storage, performance, and maintenance challenges.

ClickHouseELKLog Management
0 likes · 12 min read
How We Built a Scalable Real‑Time Log Center with ClickHouse and ELK
Efficient Ops
Efficient Ops
Jan 17, 2024 · Operations

How We Built a Scalable Cloud‑Native Log Center with ClickHouse

This article details a courier company's evolution from a simple file‑based logging system to a cloud‑native log center, describing the limitations of the original architecture, the migration to an ELK stack, subsequent challenges, and the final redesign using ClickHouse for high compression, low cost, and improved query performance.

ClickHouseELKLog Management
0 likes · 12 min read
How We Built a Scalable Cloud‑Native Log Center with ClickHouse
Open Source Linux
Open Source Linux
Jan 8, 2024 · Operations

Master Linux System Logs: Commands, Files, and GUI Tools

Learn how to view and analyze Linux system logs using command-line utilities like journalctl and dmesg, explore key log files such as /var/log/messages and /var/log/auth.log, and discover graphical tools like GNOME Logs and Logwatch for effective troubleshooting and performance monitoring.

LinuxLog Managementdmesg
0 likes · 5 min read
Master Linux System Logs: Commands, Files, and GUI Tools
Java Architect Essentials
Java Architect Essentials
Jan 5, 2024 · Operations

Common ELK Deployment Architectures and Solutions for Log Management

This article introduces the ELK stack’s core components, compares four typical deployment architectures—including Logstash‑only, Filebeat‑based, and Kafka‑enhanced setups—discusses their trade‑offs, and provides practical configurations and solutions for multiline log merging, timestamp handling, and module‑specific filtering.

ELKElasticsearchFilebeat
0 likes · 11 min read
Common ELK Deployment Architectures and Solutions for Log Management
dbaplus Community
dbaplus Community
Dec 5, 2023 · Backend Development

Taming Chaotic Logs: Practical Governance and Stack Trace Optimization

Facing uncontrolled log proliferation, the author shares a real-world case of log governance that reduces costs and boosts efficiency by categorizing logs, simplifying formats, consolidating tools, introducing a three‑layer log architecture, and applying stack‑trace reversal techniques with concrete code examples.

JavaLog Managementbackend
0 likes · 16 min read
Taming Chaotic Logs: Practical Governance and Stack Trace Optimization
Architecture and Beyond
Architecture and Beyond
Nov 25, 2023 · Operations

Effective Log Management Strategy: Standards, SDK Integration, and Lifecycle Practices

The article outlines common logging problems and presents a comprehensive six‑step strategy—including clear logging standards, systematic standard management, a unified SDK, centralized log management systems, regular standard reviews, and lifecycle deprecation—to transform chaotic logs into a reliable tool that boosts development efficiency.

Log ManagementMonitoringOperations
0 likes · 7 min read
Effective Log Management Strategy: Standards, SDK Integration, and Lifecycle Practices
Architecture and Beyond
Architecture and Beyond
Nov 25, 2023 · Operations

Designing and Implementing an Effective Log System for Internet Startups

The article explains why comprehensive logging is essential for internet startups, outlines the three stages of a log system, details log levels, required fields, best‑practice principles, collection architectures such as local files and ELK, and how collected logs support monitoring, debugging, and analytics.

ELKLog ManagementObservability
0 likes · 12 min read
Designing and Implementing an Effective Log System for Internet Startups
dbaplus Community
dbaplus Community
Nov 20, 2023 · Operations

Can VictoriaLogs Really Beat Elasticsearch, Loki, and ClickHouse? A Deep Dive

VictoriaLogs, a log‑storage system marketed as a cost‑effective, high‑performance alternative, is compared against Elasticsearch/OpenSearch, Grafana Loki, and ClickHouse, highlighting its lower RAM and disk usage, faster queries, simplified setup, and specialized features such as LogSQL, Bloom filters, and custom compression.

ClickHouseGrafana LokiLog Management
0 likes · 9 min read
Can VictoriaLogs Really Beat Elasticsearch, Loki, and ClickHouse? A Deep Dive
MaGe Linux Operations
MaGe Linux Operations
Oct 30, 2023 · Operations

Boost DevOps with Docker: Automation, Monitoring, and Log Management

This article explains how Docker integrates with DevOps practices to enhance automation, streamline continuous integration and deployment, enable comprehensive container, application, and infrastructure monitoring, and centralize log collection and analysis, providing practical code examples for building, testing, deploying, and managing services efficiently.

DevOpsLog ManagementMonitoring
0 likes · 8 min read
Boost DevOps with Docker: Automation, Monitoring, and Log Management
Selected Java Interview Questions
Selected Java Interview Questions
Sep 24, 2023 · Operations

Comparison of Six Open-Source Log Management Tools

This article reviews six open‑source log management solutions—OpenObserve, Grafana Loki, SigNoz, Graylog, Syslog‑ng, and Highlight.io—detailing their features, advantages, and drawbacks to help engineers select the most suitable tool for observability, monitoring, and cost‑effective log handling.

Log ManagementMonitoringopen-source
0 likes · 15 min read
Comparison of Six Open-Source Log Management Tools
MaGe Linux Operations
MaGe Linux Operations
Sep 13, 2023 · Operations

Top 9 Log Management Solutions Compared: Features, Pricing, Pros & Cons

This article provides a side‑by‑side comparison of nine popular log management tools—Filebeat, Graylog, LogDNA, ELK, Grafana Loki, Datadog, Logstash, Fluentd and Splunk—detailing each product's core features, pricing models, advantages and disadvantages to help you choose the right solution for your observability needs.

DatadogELKFilebeat
0 likes · 16 min read
Top 9 Log Management Solutions Compared: Features, Pricing, Pros & Cons
Ops Development Stories
Ops Development Stories
Sep 1, 2023 · Cloud Native

Why OpenObserve Beats Elasticsearch with 140× Lower Storage Costs

OpenObserve is a Rust‑based, cloud‑native observability platform that offers log, metric, and trace collection with up to 140‑times lower storage costs than Elasticsearch, supports both single‑node and HA deployments, provides a built‑in UI, and includes detailed installation and query examples for Kubernetes environments.

Log ManagementRustcloud-native
0 likes · 12 min read
Why OpenObserve Beats Elasticsearch with 140× Lower Storage Costs
Efficient Ops
Efficient Ops
Jul 16, 2023 · Operations

Mastering ELK: Deploy Architectures, Multiline Logs, and Kibana Tips

This guide explains the three main ELK deployment architectures, compares Logstash and Filebeat collectors, introduces a cache‑queue option for high‑volume logs, and provides practical solutions for multiline log merging, timestamp correction, and module‑level filtering in Kibana, helping operations teams build efficient log pipelines.

ELKElasticsearchFilebeat
0 likes · 10 min read
Mastering ELK: Deploy Architectures, Multiline Logs, and Kibana Tips
FunTester
FunTester
Jun 19, 2023 · Big Data

Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability

This article provides a comprehensive overview of Kafka's architecture and fundamental concepts, covering its overall structure, key components such as brokers, producers, consumers, topics, partitions, replicas, leader‑follower synchronization, offset handling, message storage at both logical and physical layers, as well as producer and consumer workflows, partition assignment strategies, rebalancing, log management, zero‑copy I/O, and reliability mechanisms.

Distributed SystemsKafkaLog Management
0 likes · 22 min read
Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability
Baidu Geek Talk
Baidu Geek Talk
Jun 19, 2023 · Operations

How Baidu’s Tianyan Log Service Overcomes ELK’s Scaling and Performance Limits

This article examines the challenges of logging in distributed services, compares the traditional ELK stack with Baidu's Tianyan solution, details Tianyan's architecture—including Ingest, Store, Consumer, Elastic Agent, Fleet, APM, Beats, and Disruptor‑based high‑throughput pipelines—covers resource isolation, dynamic cleanup, and best‑practice recommendations for building a scalable, low‑latency log platform.

Distributed SystemsElastic StackLog Management
0 likes · 26 min read
How Baidu’s Tianyan Log Service Overcomes ELK’s Scaling and Performance Limits
Liangxu Linux
Liangxu Linux
Jun 15, 2023 · Operations

Master Nginx Log Rotation with Logrotate: A Complete Step‑by‑Step Guide

This guide explains why nginx logs grow, introduces Logrotate as the Linux log‑management tool, walks through installation, core configuration, custom rotation rules, manual execution, compression options, time‑ and size‑based policies, hourly scheduling, and handling SELinux when rotating nginx logs.

Log Managementcronlogrotate
0 likes · 17 min read
Master Nginx Log Rotation with Logrotate: A Complete Step‑by‑Step Guide
Laravel Tech Community
Laravel Tech Community
May 23, 2023 · Operations

Comparison of Common Log Management Tools: Features, Pricing, Advantages and Disadvantages

This article provides a detailed comparative overview of nine popular log management solutions—including Filebeat, Graylog, LogDNA, ELK, Grafana Loki, Datadog, Logstash, Fluentd, and Splunk—covering their core features, pricing models, strengths, and weaknesses to help readers choose the most suitable tool for their environment.

DatadogELKFilebeat
0 likes · 14 min read
Comparison of Common Log Management Tools: Features, Pricing, Advantages and Disadvantages
Architecture Digest
Architecture Digest
Apr 10, 2023 · Operations

Comparison of Common Log Management Tools: Filebeat, Graylog, LogDNA, ELK, Loki, Datadog, Logstash, Fluentd, and Splunk

This article provides a detailed comparison of nine popular log management solutions—Filebeat, Graylog, LogDNA, ELK Stack, Grafana Loki, Datadog, Logstash, Fluentd, and Splunk—covering their core features, pricing models, advantages, and drawbacks to help readers choose the right tool for centralized logging.

ELKLog ManagementMonitoring
0 likes · 13 min read
Comparison of Common Log Management Tools: Filebeat, Graylog, LogDNA, ELK, Loki, Datadog, Logstash, Fluentd, and Splunk
vivo Internet Technology
vivo Internet Technology
Mar 22, 2023 · Mobile Development

DropBoxManagerService Source Code Analysis and Client Log File Management Design

The article examines Android's DropBoxManagerService architecture, its configurable file retention policies, naming conventions, atomic operations, and broadcast mechanisms, then applies these patterns to propose a client‑side log management system using a producer‑consumer model with prioritized temporary storage, rename‑based reporting, and FileObserver‑driven collection respecting network, data, timing, priority, and battery constraints.

AndroidDropBoxManagerServiceLog Management
0 likes · 14 min read
DropBoxManagerService Source Code Analysis and Client Log File Management Design
Architecture Digest
Architecture Digest
Mar 13, 2023 · Operations

Using Logrotate to Manage Nginx Log Files

This article explains how to install Logrotate on Linux, configure it to rotate Nginx access and error logs by size, time, or frequency, demonstrates manual rotation commands, compares copytruncate and create options, and shows how to customize cron schedules and handle SELinux permissions.

Log ManagementSystem Administrationcron
0 likes · 15 min read
Using Logrotate to Manage Nginx Log Files
21CTO
21CTO
Feb 16, 2023 · Operations

Which Log Management Tool Is Right for You? A Comprehensive Comparison of 9 Solutions

This article provides a detailed comparison of nine popular log management tools—including Filebeat, Graylog, LogDNA, ELK, Grafana Loki, Datadog, Logstash, Fluentd, and Splunk—covering their main features, pricing, advantages, and disadvantages to guide readers in selecting the most suitable solution for their needs.

ELKLog ManagementMonitoring
0 likes · 16 min read
Which Log Management Tool Is Right for You? A Comprehensive Comparison of 9 Solutions
MaGe Linux Operations
MaGe Linux Operations
Jan 19, 2023 · Operations

Mastering ELK: Compare Three Log Collection Architectures and Solve Common Issues

This article introduces the ELK stack’s core components, compares three typical deployment architectures—including Logstash‑only, Filebeat‑assisted, and Kafka‑backed designs—highlights their trade‑offs, and provides practical solutions for multiline log merging, timestamp correction, and module‑specific filtering using Logstash and Filebeat configurations.

ELKElasticsearchFilebeat
0 likes · 11 min read
Mastering ELK: Compare Three Log Collection Architectures and Solve Common Issues
Code Ape Tech Column
Code Ape Tech Column
Jan 14, 2023 · Operations

Comparison of Common Log Management Tools: Features, Pricing, Pros and Cons

This article provides a detailed comparison of nine popular log management solutions—including Filebeat, Graylog, LogDNA, the ELK stack, Grafana Loki, Datadog, Logstash, Fluentd, and Splunk—covering their main features, pricing models, advantages, and disadvantages to help readers choose the right tool for their needs.

ELKLog ManagementMonitoring
0 likes · 16 min read
Comparison of Common Log Management Tools: Features, Pricing, Pros and Cons
Data Thinking Notes
Data Thinking Notes
Dec 28, 2022 · Big Data

How NetEase Gaming Cut Data Warehouse Costs by 85%: A Data Governance Case Study

This case study details how NetEase Interactive Entertainment’s data team tackled massive log‑management chaos, storage bloat, and high overseas costs by standardizing logs, sharing a real‑time ODS layer, automating lifecycle management, tiered storage, and peak‑shaving compute scheduling, ultimately saving millions of yuan.

Log Managementstorage tiering
0 likes · 11 min read
How NetEase Gaming Cut Data Warehouse Costs by 85%: A Data Governance Case Study
NetEase Yanxuan Technology Product Team
NetEase Yanxuan Technology Product Team
Oct 17, 2022 · Big Data

Design and Evolution of a Unified Log Platform

NetEase Yanxuan built a unified log platform that consolidates collection, delivery, processing, storage, analysis, and alerting for near‑real‑time and offline logs, using Flume, a Golang‑based Loggie agent, Kafka, Flink, HBase and Elasticsearch to ensure high performance, data‑quality, container‑native isolation, full‑link traceability, and automated scaling, with Loggie now open‑sourced.

Log Managementlog collectionlog platform
0 likes · 38 min read
Design and Evolution of a Unified Log Platform
DataFunSummit
DataFunSummit
Oct 6, 2022 · Big Data

JD Big Data Log Lifecycle and Alerting Best Practices

This article presents a comprehensive overview of JD's big‑data log lifecycle, covering background, platform capabilities, log collection methods, processing functions, storage strategies, query mechanisms, DSL extensions, data delivery, and alerting techniques to help engineers build efficient and reliable log management solutions.

ELKFilebeatGELF
0 likes · 14 min read
JD Big Data Log Lifecycle and Alerting Best Practices
Architect's Guide
Architect's Guide
Aug 12, 2022 · Operations

ELK Stack Deployment Architectures, Common Issues, and Solutions

This article introduces the ELK stack, compares three typical deployment architectures—Logstash as collector, Filebeat as collector, and a cache‑queue‑enhanced design—then details practical solutions for multiline log merging, Kibana timestamp handling, and module‑based log filtering, concluding with best‑practice recommendations.

ELKFilebeatKibana
0 likes · 8 min read
ELK Stack Deployment Architectures, Common Issues, and Solutions
DataFunSummit
DataFunSummit
Jul 17, 2022 · Big Data

Elasticsearch and Big Data: Architecture, Use Cases, and Advantages

This article explains what Elasticsearch is, how it solves database acceleration, log observability, and data analysis problems, details its core components and underlying engine features, compares its strengths and weaknesses, and presents classic application scenarios and a real‑world case study integrating Elasticsearch with Flink for large‑scale log analytics.

Big DataElasticsearchFlink
0 likes · 13 min read
Elasticsearch and Big Data: Architecture, Use Cases, and Advantages