Fundamentals and Comparative Overview of Open‑Source Monitoring Systems (Zabbix, Open‑Falcon, Prometheus)

Hello everyone, I am Chen.

I've opened a private technical community on Knowledge Planet where I regularly share practical content; you can join via the link.

This article systematically organizes the basic concepts, principles, and architecture of monitoring systems, and introduces several widely used open‑source monitoring products for reference during selection. The content is divided into three parts:

Essential monitoring fundamentals

Introduction to mainstream monitoring systems

Monitoring system selection advice

Essential Monitoring Fundamentals

We can think of a monitoring system as a sentinel in ancient warfare: it issues early warnings when trouble approaches, allowing defenders to respond quickly.

For applications, monitoring acts as a third eye, helping locate problems such as a Redis outage or server memory exhaustion, enabling rapid diagnosis.

It also allows proactive alerts to prevent issues before they occur.

1. Functions of a Monitoring System

Help locate faults: Metrics help analyze and pinpoint failures.

Alert to reduce failure rate: Early warnings enable preventive actions.

Assist capacity planning: Data supports server, middleware, and cluster capacity decisions.

Assist performance tuning: JVM GC counts, response times, slow SQL, etc., can be monitored and optimized.

2. Common Monitoring Targets and Metrics

Server monitoring: CPU, memory, disk usage, I/O throughput, network traffic, etc.

MySQL monitoring: TPS, QPS, connection count, slow queries, InnoDB buffer hit rate, etc.

Redis monitoring: Memory usage, cache hit rate, key count, response latency, client connections, persistence metrics, etc.

MQ monitoring: Connection count, queue depth, production/consumption rates, message backlog, etc.

Application monitoring: HTTP interface: availability, request volume, latency, error count JVM: GC count/duration, memory region sizes, thread count, deadlocks Thread pool: active threads, queue size, execution latency, rejected tasks

3. Basic Monitoring Workflow

Data collection: Methods include log instrumentation, JMX, REST APIs, command‑line tools, or SDKs.

Data transmission: Collected data is reported via TCP, UDP, or HTTP, using push or pull models.

Data storage: Options range from relational databases (MySQL, Oracle) to time‑series databases (RRDTool, OpenTSDB, InfluxDB) or HBase.

Data visualization: Graphical dashboards display metrics.

Alerting: Flexible alert rules with support for email, SMS, IM, etc.

Comparison of Common Open‑Source Monitoring Systems

Below are three widely used open‑source monitoring solutions: Zabbix, Open‑Falcon, and Prometheus.

1. Zabbix Overview

Zabbix, created in 1998, has core components written in C and a PHP‑based web UI. It is a mature, feature‑rich solution used by roughly 70% of internet companies.

Key components:

Zabbix Server: Core C component that receives data from agents/proxies, stores it, and triggers alerts.

Zabbix Proxy: Optional component for distributed data collection, reducing load on the server.

Zabbix Agentd: Deployed on monitored hosts; supports both push and pull data collection.

Database: Stores configuration and metrics; supports MySQL, Oracle, and newer time‑series databases.

Web Server: PHP‑based GUI for visualization and alert configuration.

Zabbix Advantages :

Mature product: Extensive documentation and plugins cover most monitoring scenarios.

Rich collection methods: Supports Agent, SNMP, JMX, SSH, etc., with both push and pull.

Zabbix Disadvantages :

Requires agents on monitored hosts; all data resides in a database, leading to large storage requirements and potential bottlenecks.

2. Open‑Falcon (Xiaomi)

Open‑Falcon, open‑sourced by Xiaomi in 2015, is a Go and Python‑based enterprise‑grade monitoring system used by over 200 companies.

Its architecture builds on the server‑agent model but adds several specialized components for better scalability.

Falcon‑agent: Go‑based data collector deployed on monitored machines, gathering ~200 metrics.

Transfer: Distributes data to Graph (storage) and Judge (alert) components; also forwards to OpenTSDB.

Graph: Time‑series storage using RRDTool, capable of handling >80k writes per second.

Judge & Alarm: Real‑time alert evaluation and consolidation.

API: Provides query interface that abstracts storage details.

Open‑Falcon Advantages

Automatic collection: ~200 built‑in metrics without extra configuration.

Strong storage: RRDTool with consistent‑hash sharding for distributed time‑series storage.

Flexible data model: Tag‑based model enables multi‑dimensional aggregation and alerting.

Unified plugin management: Centralized distribution of custom scripts via HeartBeat Server.

Custom monitoring support: Proxy‑gateway enables easy application‑level metric collection.

Open‑Falcon Disadvantages

Limited monitoring types: Lacks built‑in support for common application servers such as Tomcat or Apache.

Community activity low: Few contributors and infrequent updates; extensions often required.

3. Prometheus (Next‑Generation Monitoring)

While Zabbix excels in traditional monitoring, it struggles with container environments; Prometheus was created to address this gap.

Prometheus is an open‑source monitoring and alerting toolkit released in 2015, written in Go, and backed by Google and the Kubernetes community.

Architecture:

Exporter: Exposes metrics via HTTP for Prometheus to scrape (e.g., node_exporter, mysqld_exporter).

Prometheus Server: Pulls metrics, stores them in a local time‑series database, and provides the PromQL query language.

Pushgateway: Allows short‑lived jobs to push metrics for Prometheus to scrape.

Alertmanager: Handles alert deduplication, grouping, and routing to email, WeChat, webhook, etc.

Web UI: Simple built‑in console; often paired with Grafana for richer dashboards.

Prometheus Advantages

Active community: Over 40k GitHub stars and continuous maintenance.

Efficient storage: Single binary with local disk storage, no external dependencies.

Excellent container support: Auto‑discovery of containers; native integration with Kubernetes and etcd.

Pull‑based architecture: Easy to deploy anywhere without needing agents.

Prometheus Disadvantages

Focused on metric monitoring; not suitable for logs, events, or tracing.

Pull model requires all endpoints to be reachable, demanding careful network security planning.

Large number of metrics may require pruning.

Selection Recommendations

Based on the above overview, consider the following when choosing a monitoring solution:

1. Clearly define monitoring requirements: objects, scale, and alert features.

2. Start with an open‑source solution; avoid over‑engineering a one‑size‑fits‑all platform initially.

3. For up to a few hundred nodes, Zabbix offers maturity, extensive documentation, and stability; performance can be improved with partitioning, SSDs, proxies, or push collection.

4. Zabbix excels at server monitoring but is weaker in application‑level metrics; Open‑Falcon and Prometheus handle custom instrumentation better.

5. New‑generation systems provide flexible data models, mature time‑series storage, and powerful alerting; if you lack Zabbix expertise, consider Open‑Falcon or Prometheus.

6. Open‑Falcon’s strength lies in data sharding for large scale; Prometheus is the de‑facto choice for container monitoring.

7. All three integrate smoothly with Grafana for rich visualizations.

8. Using multiple monitoring systems concurrently is common in early‑stage enterprises.

9. As scale and custom needs grow (e.g., CMDB integration), Open‑Falcon or Prometheus are more adaptable due to their APIs.

10. If you prefer to build your own solution, study these architectures for inspiration.

Feel free to join Chen’s Knowledge Planet for continuous learning and technical exchange.

Additional columns available in the Knowledge Planet include:

"Getting Into Big Tech" : interview topics, system design, performance tuning, etc.

"Billion‑Scale Data Sharding Practice" : articles and videos on large‑scale data partitioning.

"Deep Dive Spring Cloud Alibaba" : source‑level exploration of Spring Cloud Alibaba components.

"Deep Dive Spring Boot" : comprehensive Spring Boot tutorials from basics to source code.

"Deep Dive Spring" : 47+ articles covering Spring from introduction to source code.

Java backend source code explanations and full‑stack learning roadmap.

Final Note (Please Support)

Each of Chen’s articles is carefully crafted. The three columns have been compiled into PDFs; you can obtain them by following the public account and replying with the relevant keywords.

"Spring Cloud Advanced" PDF

"Spring Boot Advanced" PDF

"MyBatis Advanced" PDF

If this article helped you, please like, view, share, and bookmark—it fuels my continued effort.