Monitoring HTTP Services with Prometheus and Blackbox Exporter

HTTP (Hypertext Transfer Protocol) is a fundamental application‑layer protocol used for web data communication, and monitoring HTTP services is essential for operations teams to detect and resolve issues quickly.

The article introduces how to combine Prometheus with the open‑source Blackbox Exporter to monitor HTTP applications, providing end‑to‑end observability.

Blackbox Exporter is a Prometheus exporter that probes network services (HTTP/HTTPS, TCP, DNS, ICMP, etc.) by sending requests and exposing availability and performance metrics. Unlike other exporters, it requires defining probe modules (e.g., an HTTP module) in its configuration file.

Installation (Linux) :

$ cd /opt/
$ wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz
$ tar -xvf blackbox_exporter-0.19.0.linux-amd64.tar.gz
$ sudo cp blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter /usr/local/bin/
$ blackbox_exporter --version

Exporter configuration – create blackbox.yml defining a module named http_2xx that probes HTTP endpoints and expects a 2xx response within 5 seconds:

modules:
  http_2xx:
    prober: http
    timeout: 5s
    http:
      valid_status_codes: [200]
      valid_http_versions: ["HTTP/1.1", "HTTP/2"]
      method: GET

Start the exporter with the custom configuration:

sudo blackbox_exporter --config.file=blackbox.yml &

Prometheus job configuration – add a job named blackbox_http that uses the http_2xx module to probe target sites (e.g., Baidu and 163.com):

- job_name: 'blackbox_http'
  metrics_path: /probe
  params:
    module: [http_2xx]
  static_configs:
    - targets:
        - http://www.badiu.com
        - http://www.163.com
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 192.168.214.108:9115  # Blackbox Exporter address

After reloading Prometheus, the defined targets appear and start generating probe metrics.

Grafana visualization – import a dashboard template, select the Prometheus data source, and view real‑time HTTP probe metrics such as request duration, redirects, SSL status, and response codes.

Alert rules – use the metrics exposed by Blackbox Exporter to create alerts. Example rule triggers when the HTTP status code is greater than 399:

groups:
- name: http alert
  rules:
  - alert: HttpHealthCheckFailed
    for: 1m
    expr: probe_http_status_code > 399
    annotations:
      description: 'Http health check failed, current value: {{ $value }}'
      limit: 'Abnormal status code'

Additional alerts can be customized and managed with Alertmanager.

In summary, by integrating Prometheus, Blackbox Exporter, and Grafana, you can obtain real‑time performance data for HTTP services, visualize it, and set up alerts to promptly detect and resolve issues.