Mastering Zabbix: From Installation to Advanced Monitoring and Automation

Monitoring Overview

1.1 Website Availability

High Availability (HA) is measured by the number of nines (3‑5) representing the percentage of uptime over a year. The following calculations illustrate the downtime for each level:

<code>1 nine (90%): 36.5 days</code><code>2 nines (99%): 3.65 days</code><code>3 nines (99.9%): 8.76 hours</code><code>4 nines (99.99%): 52.6 minutes</code><code>5 nines (99.999%): 5.26 minutes</code><code>6 nines (99.9999%): 31 seconds</code>

1.2 What to Monitor

Monitor anything that can be queried via commands, including infrastructure, hardware, and services.

1.2.1 Monitoring Scope

1.3.1 Infrastructure

Remote management cards (e.g., Dell iDRAC, HP iLO, IBM IMM) enable remote server management.

1.3.2 Hardware

2. Installing Zabbix

2.1 Environment Check

2.2.1 Installation Methods

Compile installation (many services, complex environment)

Yum installation (clean environment)

Yum requires a repository mirror (e.g., http://www.cnblogs.com/clsn/p/7866643.html)

2.2.2 Server Quick‑Install Script

2.2.3 Client Quick‑Deploy Script

2.3 Connectivity Test

2.3.1 Install zabbix-get on Server

<code>yum install zabbix-get</code>

3. Web Interface Operations

3.1 Zabbix Web Installation

3.1.1 Access via Browser

http://10.0.0.6/zabbix/setup.php

During setup, view error messages for troubleshooting.

Select MySQL, enter password.

Host and port defaults are fine; customize the name.

Confirm and proceed to the next step.

Installation completes; click Finish.

Login with username Admin and password zabbix (case‑sensitive).

3.2 Adding Monitoring Items

3.2.1 Modify Zabbix Server Host

Configuration → Hosts

Host name must match the actual hostname; visible name is for UI display.

Enable the host after editing.

Host now appears in the monitoring list.

3.2.2 Add New Host

Configuration → Hosts → Create Host

Check the box to enable the host.

Attach the Linux OS template (add small then large).

Host now has two monitoring entries.

3.2.3 View Monitoring Data

Monitoring → Latest Data → filter by IP or name.

All items for the selected host are listed.

3.2.4 View Graphs

Monitoring → Graphs → select host and graph.

4. Custom Monitoring and Alerts

4.1 Custom Monitoring

Zabbix’s built‑in template Template OS Linux (Template App Zabbix Agent) provides CPU, memory, disk, and NIC metrics. Example requirement: trigger an alarm when more than three users are logged in.

4.2 Implementing Custom Monitoring

4.2.1 Custom Syntax

4.2.2 Agent Registration

4.2.3 Create Template (Web UI)

Configuration → Templates → Create Template

Add the template.

4.2.4 Create Application Set

Application set groups monitoring items.

4.2.5 Create Monitoring Item

Define the key (e.g.,

login-user

).

4.2.6 Create Trigger

Trigger fires when the item meets a condition (e.g., logged‑in users > 3).

4.2.7 Create Graph

Visualize the custom metric.

4.2.8 Link Template to Host

4.3 Alerting

4.3.1 Third‑Party Alert Platform

OneAlert (http://www.onealert.com) provides SMS, WeChat, QQ, and phone notifications with scheduling.

4.3.2 OneAlert Configuration

Add an application named zabbix in OneAlert.

WeChat alerts require following the public account.

4.3.3 Install OneAlert Agent

4.3.4 Remove OneAlert Agent

Delete script, user, user group, and action in OneAlert settings.

4.3.5 Trigger Response

Alerts appear in WeChat and email when status changes.

Note: Emails are sent only when the status changes (good→bad or bad→good).

4.4 Monitoring Visualization

4.4.1 Grafana‑Zabbix Integration

Install the Grafana‑Zabbix plugin:

<code>grafana-cli plugins install alexanderzobnin-zabbix-app</code>

After enabling the plugin, add a Zabbix data source and import Zabbix dashboards.

4.4.2 Slideshows

Use “Composite Graph → Slides” to create automatic slide presentations.

4.5 Template Sharing

Export hosts or templates from the host page and import them elsewhere.

Template repository: https://github.com/zhangyao8/zabbix-community-repos

5. Monitoring the Entire Server Fleet

5.1 Requirement

Monitor 100 existing servers with Zabbix.

5.2 Planning

Standard metrics: CPU, memory, disk, NIC. Methods to add hosts quickly:

Clone existing hosts

Auto‑registration and auto‑discovery

Use Zabbix API (curl, Python)

Develop a custom ops platform compatible with Zabbix for special services (URL, reverse proxy, etc.).

5.3 Implementation Details

5.3.1 Hardware, System, Network Monitoring

Monitor all VM nodes, switches, routers (via port traffic or SNMP).

5.3.2 Application Service Monitoring

Backup server – monitor rsync port 873 or simulate file transfer.

NFS server – monitor RPC port 111 or use

showmount -e

.

MySQL – monitor port 3306, remote login, or Zabbix MySQL template.

Web servers – monitor port 80 or HTTP status via Zabbix web checks.

URL monitoring – use Zabbix web scenario.

Reverse proxy, PPTP, NTP – monitor respective ports.

Nginx – monitor seven connection states via custom keys.

5.3.3 General Monitoring Methods

Port monitoring:

netstat/ss/lsof | wc -l

Process monitoring:

ps -ef | grep <process> | wc -l

Client‑side simulation:

curl

for HTTP, SQL queries for MySQL, set/get for Memcached.

5.4 Full‑Network Monitoring

5.4.1 Client Script for CentOS 6

5.4.2 Auto‑Discovery Rules

Add auto‑discovery rule and discovery action.

5.4.3 Backup Server Monitoring

Use

net.tcp.listen[port]

key in a new template.

5.4.4 NFS Server Monitoring

Create template using

proc.num[,, ,]

to count NFS processes.

5.4.5 MySQL Monitoring

Add MySQL key with credentials; also monitor port 3306.

<code>[root@m01 ~]# zabbix_get -s 172.16.1.51 -p 10050 -k "net.tcp.port[,3306]"</code>

5.4.6 Web Server Monitoring

Use

proc.num[,,,nginx]

and

net.tcp.port[,80]

keys.

<code>proc.num[&lt;name&gt;,&lt;user&gt;,&lt;state&gt;,&lt;cmdline&gt;] – returns process count</code><code>net.tcp.port[&lt;ip&gt;,port] – returns 0 if cannot connect, 1 if can</code>

5.4.7 URL Monitoring

Create a simple page returning

ok

and use Zabbix web scenario.

<code>echo ok >> /application/nginx/html/www/check.html</code>

<code>for ip in 7 8 9; do curl 10.0.0.$ip/check.html; done</code>

5.4.8 Reverse Proxy Monitoring

Create custom key

keep-ip

to count specific IP occurrences.

<code>UserParameter=keep-ip,ip a | grep 10.0.0.3 | wc -l</code>

<code>[root@m01 ~]# zabbix_get -s 172.16.1.5 -p 10050 -k "keep-ip"</code>

5.4.9 Nginx Connection States

Add keys to fetch

stub_status

metrics.

<code>UserParameter=nginx_active,curl -s 127.0.0.1/status | awk '/Active/ {print $NF}'</code><code>UserParameter=nginx_accepts,curl -s 127.0.0.1/status | awk 'NR==3 {print $1}'</code><code>UserParameter=nginx_handled,curl -s 127.0.0.1/status | awk 'NR==3 {print $2}'</code><code>UserParameter=nginx_requests,curl -s 127.0.0.1/status | awk 'NR==3 {print $3}'</code><code>UserParameter=nginx_reading,curl -s 127.0.0.1/status | awk 'NR==4 {print $2}'</code><code>UserParameter=nginx_writing,curl -s 127.0.0.1/status | awk 'NR==4 {print $4}'</code><code>UserParameter=nginx_waiting,curl -s 127.0.0.1/status | awk 'NR==4 {print $6}'</code>

<code>[root@m01 ~]# zabbix_get -s 172.16.1.7 -p 10050 -k "nginx_waiting"</code>

6. Auto‑Discovery and Auto‑Registration

6.1 Overview

Auto‑discovery: Zabbix Server scans the network and registers clients (high load). Auto‑registration: Zabbix Agent reports to the server (requires correct configuration).

6.2 Passive Auto‑Discovery

Configure discovery rule (e.g., Local network) and discovery action to add hosts automatically.

7. Distributed Monitoring and SNMP

7.1 Distributed Monitoring

Use Zabbix Proxy to offload load from the central server. Example topology:

<code>Zabbix Server → Zabbix Proxy → Zabbix Agents (multiple LANs)</code>

7.1.2 Proxy Setup (CentOS 7)

Install proxy, MariaDB, and configure

/etc/zabbix/zabbix_proxy.conf

(set DBPassword, Server IP, Hostname).

<code>yum -y install zabbix-proxy-mysql</code><code>yum -y install mariadb-server</code><code>systemctl start mariadb.service</code><code>create database zabbix_proxy character set utf8 collate utf8_bin;</code><code>grant all privileges on zabbix_proxy.* to zabbix@'localhost' identified by 'zabbix';</code><code>zcat /usr/share/doc/zabbix-proxy-mysql-*/schema.sql.gz | mysql -uzabbix -pzabbix zabbix_proxy</code><code>sed -i '162a DBPassword=zabbix' /etc/zabbix/zabbix_proxy.conf</code><code>sed -i 's#Server=127.0.0.1#Server=172.16.1.61#' /etc/zabbix/zabbix_proxy.conf</code><code>sed -i 's#Hostname=Zabbix proxy#Hostname=cache01#' /etc/zabbix/zabbix_proxy.conf</code><code>systemctl restart zabbix-proxy.service</code>

Update agents to point to the proxy (Server and ServerActive).

<code>sed -i 's#172.16.1.61#172.16.1.21#g' /etc/zabbix/zabbix_agentd.conf</code><code>systemctl restart zabbix-agent.service</code>

7.2 SNMP Monitoring

Install SNMP daemon for devices that cannot run a Zabbix agent.

<code>yum -y install net-snmp net-snmp-utils</code><code>sed -i '57a view systemview   included  .1' /etc/snmp/snmpd.conf</code><code>systemctl start snmpd.service</code>

Test with

snmpwalk -v 2c -c public 127.0.0.1 sysName

.

<code>SNMPv2-MIB::sysName.0 = STRING: m01</code>

Add a host in Zabbix, select an SNMP template, and the device appears with SNMP metrics.