This guide explains how to complement Prometheus white‑box monitoring with black‑box probes by deploying the Blackbox Exporter in a Kubernetes cluster, configuring ConfigMaps, Deployments, Services, and Prometheus scrape jobs for HTTP, DNS, TCP, and ICMP checks, and using annotations for automatic service discovery.
With the shift from monolithic to distributed architectures, this article compares log‑based and network‑data‑based monitoring across data sources, precision, monitoring paths, and implementation methods, concluding that network‑data monitoring offers superior real‑time insight, lower cost, and faster deployment for full‑link observability.
This tutorial walks through installing the open‑source Nightingale monitoring system using Docker, configuring its components, adding node agents, integrating with Grafana, and setting up alerting, providing all commands, configuration files, and screenshots needed for a production‑ready deployment.
This guide explains how to use Ansible to automatically install and configure Zabbix‑agent on a large number of Linux servers, covering inventory setup, a deployment script, a playbook, execution commands, and verification of the agent listening on port 10050.
This article details a systematic full‑link performance testing workflow—including background, timing, scenario design, data preparation, capacity planning, monitoring, issue analysis, and post‑test cleanup—aimed at reliably evaluating and scaling e‑commerce services during major promotional events.
This article explains how to use the Cloud Native Prometheus ecosystem—including Prometheus Server, exporters, Pushgateway, Alertmanager and Grafana—to collect, store, query and visualize Flink job metrics, providing a complete monitoring solution for production clusters.
This article outlines a comprehensive process for diagnosing and optimizing performance problems in production business systems, covering hardware, OS, database, middleware, JVM tuning, code inefficiencies, monitoring tools, and the limitations of pre‑release testing, with practical guidelines and visual references.
This article explains why Prometheus alerts may not fire or may fire unexpectedly, covering the role of the for parameter, sampling intervals, Grafana range queries, and practical steps to diagnose and fix alerting issues.
Didi’s Logi‑KafkaManager is a multi‑tenant Kafka cloud platform that consolidates dozens of clusters into a secure, isolated gateway‑driven service offering intuitive web‑based topic management, real‑time metrics visualization, automated diagnostics, quota governance and safe scaling, delivering high internal satisfaction and enterprise commercialization.
The report details a comprehensive overhaul of Puhui’s hybrid container for ride‑hailing services, introducing full‑link tracing, container reuse, offline resource caching, image WebP conversion and pre‑fetching, which together slash first‑screen load times by over 70 % and boost the 1‑second open rate from 12 % to 91 %.
This article outlines a step‑by‑step approach for identifying root causes of performance bottlenecks in production business systems, covering common scenarios such as high concurrency, data growth, hardware limits, database and middleware tuning, code inefficiencies, and the role of monitoring and APM tools.
A detailed post explains how a mis‑configured HttpClient caused thread explosion and OOM on four servers, walks through the investigation using APM metrics, clarifies keep‑alive mechanics, and presents the fix of using a singleton HttpClient with proper connection eviction.
This tutorial walks you through using Python's requests library and Zabbix Sender to retrieve Kubernetes API metrics, covering API endpoint discovery, token generation, script deployment, host configuration, and manual trigger of checks to visualize the data.
This article explores the challenges of frontend availability, introduces chaos engineering concepts, and details Alibaba's practical approach to frontend fault injection—including static resource hijacking, a safe isolated environment, monitoring integration, and a real‑world drill that demonstrates how to measure and improve detection and response capabilities.
This guide walks through the concepts of white‑box vs black‑box monitoring, explains Prometheus Blackbox Exporter capabilities, shows step‑by‑step installation, Kubernetes configuration, probe definitions for HTTP, TCP, ICMP and SSL, and provides ready‑to‑use alert rules and Grafana dashboard integration.
This article explains the background, technical architecture, real‑world case, and key takeaways of implementing gray‑scale monitoring across web, Weex, mini‑programs, and other cross‑platform front‑end solutions to improve issue detection and reduce mean time to recovery.
This article outlines Zhenkun's mobile engineering journey from 2020 to 2021, detailing the shift to componentized architecture, the adoption of Flutter hybrid development, CI automation, and real‑time monitoring to boost development efficiency and product quality.
This article outlines the golden rules for constructing Spring Boot microservices, covering monitoring with Spring Boot Admin and Grafana, exposing metrics via Actuator, centralized logging with ELK, clear API documentation using Swagger, YApi or smart‑doc, transparent build info, and keeping dependencies up‑to‑date.
This guide shows how to download, extract, and run the lightweight nmon performance monitoring tool on CentOS 7, including the exact commands to fetch the package, choose the correct binary, start the utility, and view CPU and memory statistics using interactive keys.
This guide walks through installing SkyWalking‑nginx‑lua, renaming conflicting scripts, modifying the nginx‑ingress controller’s template to inject SkyWalking environment variables and tracing buffer, building a custom Docker image, and deploying it with the required environment variables so that request traces appear in the SkyWalking UI.
This article explains the layered architecture of microservice monitoring, detailing five monitoring levels—from infrastructure to end-user experience—along with essential monitoring points such as logs, metrics, tracing, alerts, and health checks, and presents a typical monitoring stack using agents, Kafka, ELK, and InfluxDB.
This article provides a comprehensive, step‑by‑step tutorial for installing the open‑source Nightingale monitoring platform using Docker, covering code retrieval, Docker‑compose setup, node configuration, service startup, Grafana integration, and essential UI features, enabling a high‑availability, hybrid‑cloud monitoring solution.
Youzan’s log‑governance guide uses a car‑dashboard analogy to show why precise ERROR logs and sensible alerts matter, defines INFO/WARN/ERROR levels, sets daily reduction targets, leverages top‑error analysis and water‑level monitoring, and ultimately cut daily ERROR entries from thousands to about one hundred while catching issues before incidents.
This article examines common causes of increased latency in Redis—including high‑complexity commands, large keys, concentrated expirations, memory limits, fork overhead, CPU binding, AOF settings, swap usage, and network saturation—and provides practical monitoring and configuration techniques to diagnose and reduce delays.
This guide walks through building a comprehensive MySQL monitoring platform using Prometheus and Grafana, covering exporter installation, configuration, key performance metrics such as replication health, query throughput, slow queries, connection limits, buffer pool usage, and provides ready‑made Grafana dashboards and alerting rules.
The platform implements comprehensive metadata governance by extracting, standardizing, and ingesting basic, trend, resource, lineage, and task metadata from offline and real‑time systems via a Kafka‑based SDK, enabling unified storage, monitoring, alerts, and future automation to improve data asset visibility and quality.
The article details WeChat’s three‑in‑one strategy of precise product, agile projects, and robust technical support, explaining how the team achieves massive scalability, high availability, extensible protocols, resilient disaster recovery, and embedded monitoring through practices like small‑system‑big‑scale, gray‑release, and foundational components.
This article examines the challenges of moving machine‑learning models from offline training to online serving, proposes a modular architecture—including model gateway, data source gateway, business service center, monitoring, and RPC components—to enable rapid model deployment, version management, traffic mirroring, gray‑release, and real‑time monitoring.
The article explains how Home Depot (THD) adopted the VALET model—a five‑dimensional SLO language covering Volume, Availability, Latency, Error, and Ticket—to unify communication, automate data collection, and improve reliability across its massive retail and e‑commerce infrastructure.
This guide explains the core concepts of logging, metrics, and tracing in micro‑service monitoring, introduces Prometheus, exporters, and Grafana, and provides step‑by‑step instructions for downloading, installing, configuring, and visualizing metrics using these tools.
This article explains how to build a real‑time data processing platform using Flink, covering the Lambda architecture, design approaches, SQL and custom‑Jar task definitions, UI drag‑and‑drop, cluster resource management on Yarn and Kubernetes, submission modes, scheduling, permission and metadata handling, logging, and monitoring with Prometheus and Grafana.
This article describes how 58.com designed and optimized a ClickHouse‑based OLAP platform for massive user‑behavior data, covering the reasons for choosing ClickHouse, its key features, multi‑layer architecture, configuration management, automation scripts, monitoring, performance benchmarks, and future improvement plans.
This guide explains how to use Prometheus' irate function and Grafana to calculate and visualize CPU, memory, and disk usage percentages for each host, including the necessary PromQL queries and formulas in a monitoring setup.
This article walks through the transformation of a simple online supermarket from a monolithic application to a fully fledged microservice architecture, covering design principles, common pitfalls, monitoring, tracing, logging, service discovery, circuit breaking, testing strategies, and the role of service meshes.
This article explains the meaning of iostat fields, the limitations of svctm and await, how /proc/diskstats provides raw counters, and offers formulas and examples for accurately analyzing Linux disk performance.
This article examines the critical role of stability governance in evolving systems, outlines a three‑stage framework—usability, monitoring alerts, and online emergency—illustrated with a case study of an electronic waybill service, and shares concrete strategies for prevention, detection, response, and post‑mortem to achieve predictable, observable, and fast‑acting reliability.
This tutorial walks through installing Alertmanager on a Kubernetes node, configuring its SMTP settings, integrating it with Prometheus for alerting, defining alert rules, and verifying that email notifications are correctly sent when a monitored node fails.
This guide walks through installing Prometheus on the master node, deploying node_exporter on both master and worker nodes, and setting up Grafana on a third node, including service files, systemd registration, and verification of monitoring endpoints within a Kubernetes environment.
This article analyzes the concept of stability governance, outlines its five fault‑management sub‑domains, examines the pain points of an electronic waybill service, and presents a comprehensive three‑phase strategy—prevention, perception, reach, mitigation, and post‑mortem—backed by concrete implementation steps in availability, monitoring, and online emergency handling.
iQIYI’s middleware team introduced a unified microservice standard architecture—combining a single SDK, centralized infrastructure (Nacos registry, Kong gateway, Apollo config, Prometheus‑SkyWalking monitoring, ChaosBlade), the QDAS platform, and extensible open‑source practices—to eliminate redundant builds, ensure high availability, streamline governance, and pave the way for cloud‑native service‑mesh evolution.
This guide explains how to install Prometheus on a Linux server using both the binary method and a Docker container, covering downloading, extracting, configuring the prometheus.yml file, running the service, and verifying the installation.
This article explains how Netflix built the Telltale monitoring system to consolidate data sources, provide multidimensional health assessments, deliver intelligent alerts, and streamline incident management for over 100 production applications, reducing on‑call fatigue and improving service reliability.
This article introduces Xrkmonitor, a Chinese open‑source monitoring platform that combines point monitoring, log collection, data visualization, and alerting, and provides detailed advantages, featured functions, online and offline deployment guides, and the underlying technology stack.
This tutorial walks through installing and configuring Apache SkyWalking, an open‑source APM system for micro‑services and cloud‑native environments, covering its architecture, Elasticsearch and InfluxDB storage setup, agent deployment, service startup, alarm integration, and essential documentation links.
This article chronicles a senior Alibaba technologist’s decade‑long journey through operations, monitoring, resource management, and product development, sharing practical insights on system automation, team leadership, career promotion, and the mindset needed to evolve from a junior engineer to a cloud‑native solutions architect.
This article explains how to use ChaosBlade for fault injection and SkyWalking for monitoring to improve the high‑availability of distributed microservice systems, covering tool installation, experiment design, step‑by‑step execution, and real‑world case studies with detailed commands and metrics.
This guide walks through manually deploying a highly available Prometheus and Alertmanager stack on Kubernetes using StatefulSets, StorageClasses, and persistent volumes, covering environment setup, RBAC, ConfigMaps, services, node exporters, kube‑state‑metrics, and verification steps.
This article explains the cause of InfluxDB’s max‑value‑per‑tag error when Prometheus remote‑writes high‑cardinality tags, analyzes why the built‑in memory index triggers OOM protection, and presents three practical solutions—including moving indexes to disk, storing high‑cardinality tags as fields, and filtering them before write—to ensure stable monitoring data persistence.
This article introduces a curated list of Kubernetes tools—including cluster deployment solutions, monitoring utilities, CLI helpers, and development aids—explaining how each simplifies container orchestration, enhances DevOps workflows, and empowers engineers to manage, observe, and extend their Kubernetes environments efficiently.
This article shares practical experiences with Prometheus in Kubernetes, covering core principles, limitations, common exporters, metric selection, capacity planning, high‑availability strategies, query optimization, and integration with Grafana, offering actionable guidance for building reliable, scalable monitoring solutions.
This article outlines a comprehensive process for identifying root causes of performance bottlenecks in production business systems, covering hardware, database, middleware, JVM settings, code inefficiencies, and monitoring tools, and provides practical optimization techniques for each layer.
At the 2020 GOPS Global Operations Conference, Tencent’s senior operations engineer Xie Hailin detailed the design and implementation of the Panshi platform—a comprehensive, high‑availability solution that unifies change management, fault handling, continuous operation, and disaster recovery to ensure uninterrupted payment services for billions of daily transactions.
This guide compiles a comprehensive set of Linux shell commands for deleting zero‑byte files, inspecting processes, checking CPU, memory, disk usage, network load, and other system metrics, plus a collection of useful regular expressions for text processing and validation.
This article walks through the evolution of an online supermarket from a simple monolithic website to a fully split microservice architecture, highlighting the pitfalls of ad‑hoc growth, the need for service abstraction, monitoring, tracing, fault tolerance, testing, and the trade‑offs of frameworks versus service mesh.
During JD.com’s 11.11 Global Shopping Festival, the JD Cloud Log Service handled petabyte‑scale log data, delivering real‑time monitoring, cost‑effective storage, high‑availability architecture, circuit‑breaking, rate‑limiting, auto‑scaling and comprehensive dashboards to ensure stable operation of the massive traffic surge.
This article explains how to use Zabbix Agent2 to monitor Redis, covering the plugin's architecture, configuration priority, methods for retrieving INFO, CONFIG, health status, and slow‑query logs, as well as practical steps to set up the Redis template in Zabbix.
The article presents a comprehensive case study of Haodf.com's transition to a micro‑service architecture, detailing the challenges of service stability and observability, the design of a unified governance platform with log‑holographic analysis, real‑time alerts, application profiling, SLO/SLA definition, and future roadmap for capacity and reliability improvements.
This article explains how the JSTracker platform was used to build a comprehensive end‑to‑end front‑end monitoring and data analysis solution that meets the 1‑5‑10 safety production goal—detecting issues within one minute, locating them in five, and fixing them in ten—by improving coverage, subscription, metrics, and gray‑release monitoring for Alibaba’s Double‑11 promotion.
This article outlines how a city‑commercial bank redesigned its monitoring architecture using a closed‑loop “monitor‑manage‑control” strategy, detailing the current challenges, the three‑tier solution, its advantages, and future directions for automated, AI‑enhanced operations.
This guide walks you through installing Prometheus, Grafana, Alertmanager, node_exporter, cadvisor, and blackbox_exporter on CentOS 7, configuring them with Docker or binaries, setting up Prometheus scrape jobs and alert rules, and using a script to add or remove monitored targets.
In this talk, Yun Jin explains how HBase’s petabyte‑scale, horizontally‑scalable architecture—built on Hadoop, HMaster, RegionServers, and Zookeeper—enables e‑commerce platforms to handle extreme promotion‑day traffic by supporting high‑throughput reads/writes, time‑series monitoring, massive order storage, and robust HA, while covering essential table operations, monitoring, and troubleshooting techniques.
The article explains how long‑running database transactions can exhaust connection pools, block threads, and cause widespread service failures, then offers practical strategies—including keeping transactions short, removing RPC calls, enhancing monitoring, and reviewing code—to detect and prevent these high‑risk issues.
This article explains fundamental Elasticsearch concepts such as indexes, index types, and the immutable nature of shard counts, demonstrates how to define custom shard and replica settings when creating or updating indices, and outlines essential monitoring considerations and commands for cluster health.
This article reveals how JD's Zhilian Cloud prepares for the massive 11.11 shopping festival by rapidly mobilizing teams, defining protection scopes, estimating resources, implementing high‑availability across regions and AZs, applying business degradation and elastic scaling, and establishing comprehensive monitoring and rehearsal practices to ensure a smooth, resilient promotion.
This article explores the design and implementation of the Yueying front‑end monitoring SDK, covering its purpose, core design principles, module architecture, reference formats, semantic versioning, key interfaces, testing strategy, and user‑experience enhancements such as quick integration and dynamic sampling.
This article translates and expands on the author’s seven‑year experience with monitoring and alerting, presenting symptom‑based principles, practical guidelines for rule design, incident handling, and operational processes to create a robust, low‑noise alerting system.
This article presents a structured approach to mastering microservices, covering the learning pyramid concept, a detailed learning path with resource collection, and an overview of the six essential components—service description, registry, framework, monitoring, tracing, and governance—along with practical tips and visual diagrams.
This article explains what an SDK is, outlines key design principles such as minimalism, stability, and extensibility, and walks through the practical implementation of Alibaba’s Yueying front‑end monitoring SDK, covering architecture, module division, versioning, core interfaces, testing, and deployment options.
This article explains the typical reasons behind Redis latency spikes—such as complex commands, big keys, concentrated expirations, memory limits, fork overhead, CPU binding, AOF settings, swap usage, and network overload—and provides practical steps and monitoring techniques to identify and resolve each problem.
This article details ZTO Express's journey of adopting Apache Kylin for OLAP, comparing it with Presto, describing platform architecture, performance gains, integration with scheduling and monitoring systems, and the practical optimizations and future plans that enabled sub‑second query responses on massive daily data volumes.
This article explains common JVM problems such as OutOfMemoryError, memory leaks, and thread deadlocks, then introduces core monitoring tools—jps, jstack, jmap/jhat, jstat, and hprof—detailing their syntax, options, and practical examples to help Java developers diagnose and tune production applications.
The article analyzes a persistent active alert observed via Thanos Ruler's HTTP interface, identifies the buffering queue bottleneck as the root cause, and proposes adjusting the maxBatchSize parameter to prevent alert delay and automatic resolution failures.
This guide walks through the inner workings of Zabbix Agent2’s Docker monitoring plugin, detailing how it communicates via the Docker API, the key configuration files, the query mechanism, and how to apply the provided templates to automatically discover and display container and image metrics.
This article systematically explains monitoring fundamentals, the seven core functions of a monitoring system, proper usage practices, common monitoring objects and metrics, the basic data flow, and provides detailed comparisons of three popular open‑source solutions—Zabbix, Open‑Falcon, and Prometheus—to guide informed selection decisions.
This article explores Prometheus as a cloud‑native monitoring solution, covering core principles, limitations, metric selection, exporter consolidation, Kubernetes deployment nuances, memory and storage planning, high‑availability designs, and advanced features like rate calculations, cardinality management, and predictive alerts.
This article explains what Druid Monitor and Druid Admin are, why cluster‑level monitoring is needed in microservice architectures, and provides a complete Spring Cloud starter implementation with configuration examples, code snippets, and usage limitations.
This article explains how Prometheus uses a time‑series database (TSDB) to handle massive monitoring data, detailing its concepts, query examples, storage engine design, indexing mechanisms, and the benefits of pre‑computing expressions for efficient real‑time analysis.
The article presents iQIYI’s service‑maturity model for micro‑services, outlines how scores across development, deployment and operation stages reveal common deficiencies such as code style, testing, gray‑release and alert handling, and recommends concrete optimization practices—including unified coding standards, automated testing, robust rollback, circuit‑breaking, monitoring, and emergency procedures—to raise services to mature, high‑scoring levels.
The article details a real‑world investigation of a Java HTTP service that experienced a sudden QPS increase and response‑time spikes, tracing the issue through database queries, local method latency, CPU load, frequent ParNew GCs, and large response payloads, and presents concrete remediation steps.
This guide surveys over a hundred modern frameworks and tools across fourteen critical backend domains—message queues, caching, sharding, data sync, communication, micro‑services, distributed utilities, monitoring, scheduling, entry proxies, storage, CI/CD, debugging, and local utilities—offering concise recommendations and practical insights for architects and engineers.
Meituan’s AIOps initiative replaces manual rule‑based monitoring with the Horae platform, which automatically classifies time‑series metrics, applies CNN and XGBoost models to detect periodic anomalies, achieves over 90 % precision in production, and paves the way for broader metric types, forecasting, and advanced fault‑localization.
This article explains what etcd is, outlines common failure points, and provides step‑by‑step instructions for collecting etcd metrics via curl, configuring Prometheus scraping, creating alerts, and using Sysdig Monitor to observe key health indicators in a Kubernetes environment.
A concise collection of Linux command‑line snippets helps you query Oracle client IPs, kill specific processes, count connections, summarize traffic, find large files, measure copy time, and monitor CPU and memory usage, all useful for DB and system administrators.
This guide walks through a real‑world call‑center slowdown incident, outlines common fault‑handling techniques, proposes monitoring enhancements, details a comprehensive emergency‑response plan, and introduces intelligent event‑processing concepts to help operations teams resolve outages faster and more reliably.
Performance optimization is a continuous, data‑driven practice that monitors response time and concurrency, applies techniques such as indexing, caching, parallelism, and asynchronous processing, and in Youzan’s live‑streaming product‑detail case reduced bottlenecks by adding multi‑level caches, circuit‑breaker fallbacks, and parallel sub‑task aggregation.
Sampler is a lightweight tool that runs shell commands, visualizes their output, and can trigger alerts; configured via simple YAML, it works on macOS, Linux and Windows, supports various components such as runcharts, sparklines, gauges, and interactive shells for monitoring databases, queues and system metrics.
This guide explains the most common Prometheus Operator metrics for Kubernetes, detailing each metric's purpose, the PromQL expression to monitor it, and the related underlying metrics, helping you fine‑tune alerts and avoid unnecessary noise in your cluster monitoring.
This article explains why Redis can become slow, covering typical latency causes such as high‑complexity commands, large keys, concentrated expirations, memory limits, fork overhead, CPU binding, AOF settings, swap usage, and network saturation, and provides practical troubleshooting steps and best‑practice recommendations.
This article explains Prometheus's built‑in time‑series database (TSDB), covering its concepts, storage configuration, block structure, write‑ahead log, mmap reads, inverted indexing, data compression, and remote storage integration for scalable monitoring.
Version 3.20.09.0 of the open‑source DTLE data‑transfer component for MySQL has been released, introducing replication‑delay and memory‑usage monitoring with Prometheus, providing configuration examples and Docker commands, and fixing incremental serialization, CPU usage, and uppercase‑where clause handling.
The article presents a design for an asynchronous component that is monitorable, fault‑tolerant, and integrates with zero overhead, compares Akka, RxJava, and a custom JUC‑based implementation, and selects the latter—using extended Callables and a CountDownLatch—to track business units, handle timeouts, and provide fallback behavior.
This article explains how to design and implement a Docker container monitoring system using CAdvisor for metric collection, InfluxDB for time‑series storage, and Grafana for visualization, covering deployment, integration, common issues, and practical configuration details.
A comprehensive Q&A with SRE experts explores how Zabbix and Prometheus compare across scalability, storage, alert handling, intelligent monitoring, dashboard design, automation, migration strategies, and performance‑cost trade‑offs for modern infrastructure.
iQIYI’s payment team built a full‑chain load‑testing framework that isolates data, mocks dependencies, constructs realistic multi‑service traffic, and executes protected tests to expose bottlenecks, guide scaling and optimizations, and ultimately ensure reliable payment services during traffic spikes, while planning a unified automation platform.
This article walks through the evolution of an online supermarket from a simple monolithic web application to a fully decomposed microservice architecture, highlighting the challenges of scaling, the need for monitoring, tracing, service discovery, fault tolerance, and the eventual adoption of a service mesh.
The article outlines the complex nature of testing environments, enumerates typical problems such as resource constraints, external dependencies, and service bugs, and presents systematic troubleshooting methods, useful tools, and real‑world case studies to improve reliability and efficiency.
This article examines the complexities of testing environments, outlines typical causes of failures such as resource constraints, external dependencies, and service bugs, and provides systematic troubleshooting methods, useful tools, and real‑world case studies to improve reliability and efficiency.
This article explains how to monitor Kafka consumer groups using the built‑in kafka‑consumer‑groups tool and the Kafka Manager UI, providing commands, field explanations, and setup steps to ensure real‑time data availability for downstream services such as MongoDB or Elasticsearch.
The article explains Linux shared‑memory fundamentals, why it outperforms file‑based IPC, demonstrates the mmap() system call, and walks through a complete Go implementation that creates, synchronizes, reads, and protobuf‑serializes advertising‑tracking metrics in the VCS monitoring platform.
This article introduces JD Cloud’s self‑built time‑series database HoraeDB, explaining its core concepts, typical use cases, architectural layers, high‑performance features, down‑sampling strategies, compression techniques, and stability measures for handling massive, 24‑hour monitoring data at scale.
