The article shares eight authentic production incident cases—from frequent JVM Full GC and memory leaks to cache avalanches, DNS hijacking, and database deadlocks—detailing their root causes, diagnostic steps, code snippets, and practical remediation strategies for engineers facing similar challenges.
This article introduces four widely used Linux monitoring utilities—iotop, htop, IPTraf, and Monit—explaining their core features and how they help quickly diagnose system performance and network issues for effective operations management.
This article explains the fundamentals of CI/CD within DevOps, compares leading CI/CD tools, outlines a four‑stage implementation process, and details the numerous benefits such as faster delivery, higher quality, improved productivity, and reduced costs.
The article uses the OVH data‑center fire as a stark reminder that startups must design robust data disaster‑recovery strategies, explaining why backups, off‑site storage, and proper architectural planning are essential to prevent catastrophic data loss and potential business collapse.
The article analyzes the instability of a company's Kubernetes clusters, identifies root causes such as unstable release processes, lack of monitoring, logging, and documentation, and proposes comprehensive solutions including a Kubernetes‑centric CI/CD pipeline, federated Prometheus monitoring, Elasticsearch logging, centralized documentation, and integrated traffic management with Kong and Istio.
The article provides a comprehensive overview of Site Reliability Engineering (SRE), explaining its origins, definition by Google, required skill sets, typical responsibilities, tools used, and how the role has evolved within DevOps and modern cloud‑native environments.
Learn step-by-step how to create a custom Zabbix Agent2 plugin in Go that retrieves HTTPS certificate expiration dates, compile and install the plugin, configure Zabbix, and set up monitoring items and alerts to proactively avoid service disruptions caused by expired certificates.
This article examines Yanxuan's inventory management challenges and presents a layered design—warehouse, physical, and sales layers—combined with a flexible lock‑stock pool to support multi‑channel, multi‑warehouse operations while addressing promotional and special‑use inventory requirements.
This guide walks you through essential Huawei switch commands, covering user and view modes, VLAN creation, port link types, batch operations, initial login procedures, service activation, link aggregation, DHCP setup, and flow‑control configuration, all illustrated with step‑by‑step screenshots.
Learn how to monitor Linux server performance using essential tools—top, vmstat, pidstat, iostat, netstat, sar, and tcpdump—understanding CPU, memory, disk I/O, and network metrics, interpreting their outputs, and applying insights to diagnose and troubleshoot system issues effectively.
This article outlines Zhuanzhuan's systematic approach to collecting, classifying, and resolving online user issues through weekly QA‑led meetings, detailed problem categorization, and targeted solutions, ultimately improving product design, reducing repetitive work, and enhancing overall operational efficiency.
Datadog, a cloud‑native monitoring platform, has surged from a modest startup to a $30‑billion market cap by expanding its product suite, leveraging strong probe technology, and capitalizing on the booming cloud‑monitoring market, with financials and customer retention indicating robust growth prospects.
A production RocketMQ cluster suffered a 10‑minute message‑send timeout after a broker’s memory failure caused a server reboot, and the post analyzes the routing registration, name‑server removal logic, client‑side timeout handling, root cause of name‑server “dead‑lock”, and proposes deployment and code‑level fixes to prevent recurrence.
The article details how 58.com’s annual Spring recruitment campaign was strategically designed with a two‑phase timeline, unified interaction framework, and immersive storytelling to drive new user acquisition, sustained activation, and significant traffic growth during the post‑holiday hiring surge.
This article explores practical ways to improve operational efficiency by examining process optimization, tool adoption, quality considerations, and engineering practices, highlighting real-world examples like OA, CICD, Spring Cloud, Java, and Kubernetes while emphasizing shared value and cultural factors.
This article details Baidu's Feed recommendation system architecture, explaining how a combination of dynamic retry scheduling, real‑time stop‑loss mechanisms, multi‑recall frameworks, ranking layer fallbacks, and IDC‑level multi‑master designs collectively ensure five‑nine availability across billions of daily requests.
Amid microservice and distributed architectures, this article explains how to design a comprehensive log analysis system—covering collection, storage, consumption, key data points, collection methods, and practical use cases like automated test generation, issue localization, and real‑time exception monitoring—to ensure system stability.
The article examines how the rapid adoption of cloud computing, DevOps, AIOps, and FinOps is reshaping the role of IT operations, highlighting new trends, evolving work boundaries, and the essential characteristics of a modern, automated, secure, and cost‑optimized operations system.
The article outlines six key DevSecOps best practices—including establishing a security model, enforcing governance policies, automating security tasks, training developers, applying network segmentation, and limiting administrative privileges—to help organizations overcome staffing and collaboration challenges while maintaining consistent security throughout the development and operations lifecycle.
This article introduces OCTO 2.0, Meituan’s next‑generation distributed service‑governance platform, detailing its overall architecture, mesh‑related features such as traffic hijacking, service subscription, lossless hot‑restart, data‑plane operations, and future cloud‑native evolution plans.
This comprehensive guide explains MongoDB cluster components, common use cases, monitoring commands, basic operational tasks, data migration procedures, troubleshooting of production issues, and optimization recommendations to achieve high performance and scalability.
This article provides an in‑depth overview of monitoring, covering its purpose, key objectives, practical methods, core processes, a detailed comparison of popular monitoring tools such as Zabbix and Prometheus, and best‑practice recommendations for building scalable, reliable, and intelligent monitoring platforms.
This article explains the Linux uptime command, describing its purpose, basic syntax, available options, common usage examples, and deeper concepts such as system load average, CPU core considerations, and how to retrieve related information from /proc files.
This guide offers a practical overview of essential Linux networking tools—including netstat, ss, sar, iftop, and tcpdump—explaining how to monitor connections, analyze traffic, capture packets, and tune kernel parameters to handle massive connection loads efficiently.
When disk usage reported by df doesn't match du and inodes are not full, the discrepancy often stems from files that were deleted while still held open by running processes, and this guide explains how to identify those processes and safely free the space.
This guide walks you through essential Linux automation techniques—including boot‑time service startup with chkconfig and rc.d, one‑off scheduling using at, and recurring jobs with crontab and shell scripts—so you can manage web servers efficiently without manual intervention.
This article outlines Youku's end‑to‑end service‑side quality assurance framework, detailing the factors that affect quality across the development lifecycle, the automated testing practices integrated into the release pipeline, the platform capabilities built for data collection and replay, and the metrics used to measure improvements in reliability and development efficiency.
This article presents a Bash script that uses a case statement to define functions for starting, stopping, reloading, and checking the status of Nginx, explaining each command and providing the full script for practical service management.
The article explains how transformational leadership—characterized by vision, inspirational communication, intellectual stimulation, supportive behavior, and individualized recognition—drives high‑performing DevOps teams, linking leadership traits to software delivery outcomes and offering practical guidance for managers to foster a strong engineering culture.
This article explains how Facebook’s Twine system orchestrates containers across millions of servers, detailing its architecture, support for stateful services, cross‑data‑center control, elastic capacity handling, and the lessons learned from eight years of large‑scale operations.
This comprehensive guide explains the purpose, models, and methods of monitoring across the entire software lifecycle, compares health checks, logging, tracing, and metric collection, and details practical implementations using tools like ELK, SkyWalking, and Prometheus for cloud‑native operations.
In a keynote at MWC 2021 Shanghai, Huawei’s Dr. Che Haiping explained how the Autonomous Networks initiative, likened to Industry 4.0, is reshaping telecom production, enabling flexible, intent‑driven services and accelerating digital transformation across the sector.
This article explains the concept of resource packages, compares several implementation approaches, and shows how they allow a single configuration to instantly update static web assets for diverse and time‑sensitive front‑end scenarios without requiring a new release.
This article analyzes the rapid growth of O2O home‑delivery, examines the challenges of delivery fulfillment, compares third‑party and self‑built rider models, and presents hybrid, batch‑ordering, and AI‑driven optimization strategies to reduce costs and boost efficiency.
An Operations Workbench built on a micro‑frontend architecture unifies fragmented e‑commerce tools into standardized Process Units and SOPs, boosting operational efficiency, reducing technical duplication, and providing data‑driven, consistent experiences for thousands of operators across hundreds of business scenarios.
This article examines the design of the "Spring Heat Value Competition" activity for a local live‑dating platform, detailing how game‑like mechanics, emotional curves, and visual styling were used to strengthen host‑user relationships, increase user participation, and drive a 116% rise in gift revenue.
This article presents a comprehensive guide to Redis monitoring and alerting, covering metric classification, threshold settings, client traffic collection, host resource usage, instance health checks, cluster failover diagnostics, and detailed explanations of Redis INFO sections with practical code examples.
This article walks through the design, scenario planning, and Java implementation of a fixed‑thread load test that simulates teacher login, knowledge‑point queries, course recommendations, and collect/uncollect actions, then presents the resulting performance metrics.
This article introduces the fundamentals of full‑chain load testing, explains why it is essential for large‑scale distributed systems, outlines the DESP model with its four simulation dimensions, and presents a detailed case study of New Oriental's continuation‑class platform including architecture, data preparation, load design, automation and recruitment information.
This article explains why Tomcat log files inherit restrictive permissions after deployment, describes Linux UMASK behavior, and provides step‑by‑step instructions to modify the catalina.sh script so that log files use the system's default 644/640 permissions.
This article defines bug tracking, outlines essential workflow steps and report contents, and evaluates a range of bug tracking tools—including BugHerd, Bugzilla, MantisBT, DebugMe, Donedone, Marker.io, Jira, Bughost, Zoho, Backlog, and Redmine—highlighting their features, integrations, pricing, advantages, and drawbacks to help teams choose the right solution.
This article explains the purpose of swap partitions, shows how to check current swap usage, and provides step‑by‑step instructions for creating swap space both via a dedicated disk partition and by using a swap file on a Linux server.
This article explains the concept of gray release, outlines a simple architecture with essential components, describes common routing strategies, and shows how to implement gray releases using Nginx, gateway services, and complex multi‑service scenarios to ensure controlled, low‑risk deployments.
This article explains how the presence or absence of a trailing slash in an Nginx proxy_pass directive affects request routing, demonstrates correct configurations with examples, and provides additional proxy and upstream settings for reliable backend forwarding.
This article outlines five best‑practice steps—including considering cluster architecture, planning recovery, simplifying operations, ensuring security, and leveraging Kubernetes portability—to help organizations reliably back up applications and data in Kubernetes environments.
This guide walks you through using the systemctl tool on modern Linux distributions to start, stop, restart, reload, enable, disable, and query services, manage system power, work with targets, handle remote hosts, and leverage related utilities like journalctl, systemd-analyze, and hostnamectl.
The article explains what digital transformation truly means for enterprises, outlines its three development stages, describes the core connection‑data‑intelligence framework, compares internal capability rebuilding with external ecosystem integration, and offers practical guidance on why and how companies should embark on digital transformation.
This guide shows how to prevent a Bash script from being executed multiple times by detecting existing instances, using lock files with process checks, and employing the flock command for reliable atomic locking, complete with practical code examples and pitfalls to avoid.
This guide demonstrates how to navigate to the RocketMQ bin directory and use various mqadmin CLI commands to list available commands, check help, monitor producer and consumer connections, query topic status, retrieve messages by offset or ID, and shut down the NameServer and brokers.
This guide explains three ways to monitor Ceph distributed storage using Zabbix—Agent2 with the RESTful module, Zabbix Sender, and custom scripts—providing step‑by‑step commands, configuration tips, and troubleshooting notes for reliable operations.
NetEase Cloud Communication’s service monitoring platform leverages data collection, preprocessing, alerting, and visualization pipelines—using HTTP APIs, Kafka, custom scripts, and NTSDB—to provide real-time insights, ensure stability, and support scalable, high‑throughput audio‑video services.
This article explains the role of Site Reliability Engineering (SRE) in bridging product and foundational technology development, outlines the software lifecycle, describes how SRE ensures system stability through controllability, observability, and protection, and provides practical best‑practice checklists and maturity levels for evaluating and improving reliability.
This article walks through a real‑world Kafka incident where a sudden surge in client connections caused ISR churn, frequent connection drops, and high response time, and explains how monitoring thread idle rates and increasing network and I/O thread counts restored stability.
This guide walks through installing GitLab CE on CentOS 8 via an RPM package, adjusting the gitlab.rb configuration, managing the GitLab service, testing access, and optionally connecting GitLab to an external PostgreSQL database running in Docker.
This article explains how internet companies can reduce soaring manual operations costs by applying intelligent monitoring techniques—such as pattern recognition and statistical anomaly detection—to automatically identify abnormal nodes among thousands of servers, streamline fault diagnosis, and improve service quality.
This guide walks through downloading the Jira Server installer, setting up a PostgreSQL database, executing the installation, opening required ports, adding the Atlassian agent for licensing, and completing initial configuration to get Jira Software 8.13.2 running on a Linux server.
This guide walks you through installing the interactive TLDR client Tldr++, explains its usage for quickly learning Linux commands, and demonstrates a complete workflow of downloading, extracting, and moving files—all without leaving the terminal.
This article explains how to perform URL redirection with Nginx by using the rewrite module for changing the browser address and the location/proxy_pass combination for keeping the address unchanged, including flag options and two practical configuration examples.
This article outlines comprehensive production‑grade Kubernetes best practices—including health probes, RBAC, resource management, network policies, monitoring, autoscaling, image security, and zero‑downtime strategies—to help teams run secure, efficient, and highly available workloads.
This tutorial walks through preparing three Kubernetes nodes, extracting and distributing Zookeeper, configuring its zoo.cfg and myid files, starting and verifying the Zookeeper ensemble, then installing Kafka, adjusting its server.properties, and finally launching Kafka across the cluster.
The Windows‑only baulk package manager version 2.0 adds experimental untar and unzip sub‑commands, detailing untar's support for various tar formats and unzip's robust baulk::archive::zip implementation that handles many compression methods, filename encoding detection, SIMD‑accelerated decompression, and strict path security.
This article shares practical insights from a technical leader on designing robust system architecture, implementing comprehensive capacity planning, establishing reliable operations processes, strengthening security, and cultivating team awareness to achieve long‑term stability for large‑scale internet services.
StackStorm is an open‑source automation platform that connects existing infrastructure and applications, enabling event‑driven actions, workflow orchestration, and rule‑based operations across services such as monitoring, deployment, and incident response.
This article explains the concept, purpose, design principles, core features, authentication methods, deployment options, and popular open‑source and commercial solutions of bastion hosts, highlighting how they centralize access control, audit operations, and improve overall IT security and compliance.
This article defines DevOps, outlines its benefits and drawbacks, and explains key concepts such as automation, CI/CD, multi‑environment deployments, early failure detection, rollback, policy enforcement, and observability, showing how they collectively improve software delivery and organizational collaboration.
This tutorial walks through installing the open‑source Nightingale monitoring system using Docker, configuring its components, adding node agents, integrating with Grafana, and setting up alerting, providing all commands, configuration files, and screenshots needed for a production‑ready deployment.
This guide explains how to use Ansible to automatically install and configure Zabbix‑agent on a large number of Linux servers, covering inventory setup, a deployment script, a playbook, execution commands, and verification of the agent listening on port 10050.
This article details a systematic full‑link performance testing workflow—including background, timing, scenario design, data preparation, capacity planning, monitoring, issue analysis, and post‑test cleanup—aimed at reliably evaluating and scaling e‑commerce services during major promotional events.
This article explains the definitions, calculations, and practical importance of MTTR, MTBF, and MTTF for reliability engineering, showing how accurate data and proper metric use enable SRE teams to improve system availability, plan maintenance, and reduce downtime.
This article clarifies the distinction between stress testing and load testing, describing their respective models—fixed‑thread and fixed‑QPS—along with key metrics, formulas, and practical benefits for evaluating system performance under varying workloads.
Learn how to securely copy files between local and remote machines using the scp command, covering basic syntax, authentication options, copying to and from remote hosts, recursive directory transfers, and transferring between two remote servers, all illustrated with practical examples.
This article details iQIYI's end‑to‑end load‑testing methodology for its payment platform, covering problem identification, core‑link mapping, environment setup, realistic traffic modeling, execution safeguards, results from capacity verification and stress testing, and future plans for a unified testing solution.
This article examines iQIYI's low-code Quqiao platform for data synchronization, detailing the software engineering background, business data flow, development challenges, research of existing solutions, platform architecture, key features, and future improvement directions.
This guide explains how to delete inconsistent Kubernetes rc, deployment, and service objects, reset etcd data, address apiserver start failures caused by missing ServiceAccount certificates, disable SELinux for fluentd logs, generate ServiceAccount keys, recover from etcd startup errors, configure host trust, change hostnames, enable VirtualBox copy‑paste, force‑delete pods and namespaces, and avoid resource‑request‑only containers causing contention.
This guide walks through installing MariaDB, creating a Gogs database and user, setting up Git, extracting and launching Gogs, and configuring the application—including disabling registration—to complete a simple self‑hosted Git server on Linux.
This article explains the concept, purpose, design principles, functional modules, authentication methods, deployment options, and open‑source implementations of bastion hosts, highlighting how they centralize control, audit, and protect privileged access to servers and network devices.
This article explains Nginx’s core concepts—including reverse proxy, load balancing, static‑dynamic separation, common commands, configuration blocks, and high‑availability setup with Keepalived—providing step‑by‑step guidance and practical examples for building robust web infrastructure.
This guide walks through the complete process of adding a new disk to a Linux server running on VMware, including disk addition, partitioning, formatting, mounting, and handling disks larger than 2 TB with detailed command examples.
This article describes how NetEase built a real‑time log intelligent classification system using Flink and AI algorithms, detailing the challenges of massive log volumes, the Drain template‑extraction method, algorithm workflow, performance results, and a practical case study that demonstrates reduced alert storms and faster issue diagnosis.
This article explains the background, technical architecture, real‑world case, and key takeaways of implementing gray‑scale monitoring across web, Weex, mini‑programs, and other cross‑platform front‑end solutions to improve issue detection and reduce mean time to recovery.
This article details ICBC's extensive MySQL deployment—nearly ten thousand nodes supporting core A‑level applications—and outlines the bank's governance framework, including current challenges, standardized operational procedures, automation, containerization, and future self‑healing strategies to ensure reliable, high‑performance database services.
The article explains Microsoft’s Test‑In‑Production (TIP) approach, describing why production is the only true environment, how they use gradual releases, feature flags, telemetry, fault injection, circuit‑breaker testing, and chaos engineering to improve reliability, micro‑service compatibility, and business continuity.
This article explains what data centers are, outlines their core components—compute, storage, and networking—covers architectural decisions, industry standards, and emerging technologies such as edge computing, micro‑data centers, cloud integration, SDN, HCI, containers, NVMe, and GPU acceleration, highlighting their impact on modern enterprise operations.
The article explains the importance of microservice monitoring and presents a five‑level monitoring hierarchy—from infrastructure to end‑user experience—along with five essential monitoring aspects and a typical architecture using agents, message queues, ELK, and time‑series databases to ensure reliable, observable services.
This article explains the concept of gray release, outlines a simple architecture with essential components, describes common strategies such as header, cookie, and parameter based routing, and provides detailed implementation guidance for Nginx, gateway, and complex multi‑service scenarios.
This guide explains how to set up Jenkins email notifications, covering both the basic system‑wide configuration via System Settings and the advanced per‑project setup using Editable Email Notification, with step‑by‑step instructions and screenshots.
This guide shows Linux commands for quickly identifying the ten processes that consume the most CPU, memory, or virtual memory, explains the key ps fields, and provides alternative sorting options for system monitoring.
This article explains what DevOps is, outlines its key benefits such as faster feedback loops, higher quality and reduced costs, compares it with traditional IT operations, and provides practical guidance on mindset shifts, infrastructure as code, automation, and tool selection for successful adoption.
This article presents a comprehensive overview of Site Reliability Engineering (SRE) as shared by Google SRE expert Ramón Medrano Llamas, covering SRE fundamentals, a typical day’s workflow, design principles for massive scale, fault‑tolerant architecture, monitoring, SLI/SLO metrics, redundancy strategies, disaster recovery, and operational best practices.
This article explains the layered architecture of microservice monitoring, detailing five monitoring levels—from infrastructure to end-user experience—along with essential monitoring points such as logs, metrics, tracing, alerts, and health checks, and presents a typical monitoring stack using agents, Kafka, ELK, and InfluxDB.
This article describes how 58.com introduced ClickHouse to handle massive daily user‑behavior logs, detailing its features, multi‑layer architecture, configuration management, monitoring, performance optimizations, and operational automation to build a high‑availability, low‑cost analytics platform.
Youzan’s log‑governance guide uses a car‑dashboard analogy to show why precise ERROR logs and sensible alerts matter, defines INFO/WARN/ERROR levels, sets daily reduction targets, leverages top‑error analysis and water‑level monitoring, and ultimately cut daily ERROR entries from thousands to about one hundred while catching issues before incidents.
This tutorial walks you through downloading Jenkins, setting up JDK 1.8, installing and configuring Supervisord, creating the Jenkins directory, configuring supervisord to manage Jenkins, retrieving the initial admin password, and completing the web UI setup, all with clear command examples and screenshots.
A field investigation of the North‑Pavilion village in Guangzhou’s university town reveals hotel room rates, occupancy patterns, and cost structures, culminating in a detailed profit calculation that shows a 15‑room hostel can generate roughly 100,000 RMB annually.
Netflix built the in‑house Telltale system to consolidate monitoring data, reduce alert fatigue, and provide intelligent, multi‑dimensional health assessments for over a hundred production applications, enabling faster incident resolution and more reliable streaming for its 200 million users.
The article outlines essential DevOps metrics—such as deployment frequency, deployment time, automated test pass rate, code commit volume, defect escape rate, cost, failure rates, detection time, unplanned work, MTTF, application performance, MTTD, MTTR, delivery time, change quality, and customer feedback—to help teams monitor and improve software delivery speed, quality, and reliability.
The article details WeChat’s three‑in‑one strategy of precise product, agile projects, and robust technical support, explaining how the team achieves massive scalability, high availability, extensible protocols, resilient disaster recovery, and embedded monitoring through practices like small‑system‑big‑scale, gray‑release, and foundational components.
This article analyses the shortcomings of traditional log‑collection agents, compares Flume and Filebeat based on low‑cost, stability, efficiency and lightweight criteria, and presents practical solutions for file discovery, offset tracking, multi‑line handling and performance tuning in modern logging pipelines.
Xianyu’s new Message Quality Platform links client, API, and server logs by a unique messageId, cleans and clusters real‑time telemetry, correlates user behavior, and visualizes abnormal nodes, giving end‑to‑end traceability that cuts incident investigation time by over 90 % and can be applied to other pipelines.
This article summarizes Tao Hui's 2020 GOPS Global Operations Conference talk, covering practical techniques for optimizing basic resources, improving network efficiency, reducing request latency, and scaling system concurrency to achieve higher throughput and lower latency in modern distributed services.
