Step-by-Step Guide to Install and Configure Alertmanager with Prometheus on Kubernetes

(1) Install Alertmanager

On the k8s-node2 server, extract the package and copy the binaries:

tar -zxvf alertmanager-0.21.0.linux-amd64.tar.gz

cp alertmanager-0.21.0.linux-amd64/alertmanager /usr/local/bin/

cp alertmanager-0.21.0.linux-amd64/amtool /usr/local/bin/

Verify the installation with alertmanager --version .

(2) Configure Alertmanager

Create the configuration directory and copy the default config file:

mkdir -pv /etc/alertmanager

cp alertmanager-0.21.0.linux-amd64/alertmanager.yml /etc/alertmanager/

Open /etc/alertmanager/alertmanager.yml and edit the SMTP and routing sections (example values shown):

global:

smtp_smarthost: 'smtp.qq.com:25'

smtp_from: '[email protected]'

smtp_auth_username: '[email protected]'

smtp_auth_password: '******'

smtp_require_tls: false

route:

receiver: mail

receivers:

- name: 'mail'

email_configs:

- to: '[email protected]'

(3) Restart Alertmanager

Run Alertmanager with the custom configuration file:

alertmanager --config.file /etc/alertmanager/alertmanager.yml

(4) Add Alertmanager to Prometheus

Append the following to prometheus.yml :

alerting:

alertmanagers:

- static_configs:

- targets:

- 192.168.20.43:9093

(5) Scrape Alertmanager metrics

Add a job definition:

- job_name: 'alertmanager'

static_configs:

- targets: ['192.168.20.43:9093']

(6) Define an alert rule

Create a rule file (e.g., node_up_rules.yml ) with the following content:

groups:

- name: node-up

rules:

- alert: node-up

expr: up{job="k8s-node1"} == 0

for: 15s

labels:

severity: 1

team: node

annotations:

summary: "{{ $labels.instance }} 已停止运行超过 15s ！"

(7) Include rule files in Prometheus configuration

Add to prometheus.yml :

rule_files:

- "rules/*_rules.yml"

- "rules/*_alerts.yml"

(8) Verify rule loading

Check Prometheus UI to ensure the new rule appears (screenshot omitted).

(9) Test the alert

Stop the k8s-node1 node; an alert should fire and appear in Alertmanager.

Alertmanager shows a pending alert:

(10) View alerts in Alertmanager UI

(11) Check email notifications

With these steps, Prometheus, Grafana, and Alertmanager are fully integrated and email alerts are functional.