Using Supervisor for Process Monitoring and Management on Linux

Supervisor is a client/server process monitoring and management tool for UNIX-like systems that ensures the stability of persistent background processes such as API services or consumers.

The article outlines typical deployment scenarios where traditional nohup cmd & fails to guarantee process stability, and introduces Supervisor as a solution.

1. Overview

Supervisor monitors target processes and automatically restarts them upon unexpected termination, supporting grouped management of multiple processes.

2. Installation

pip install supervisor

Note: Supervisor does not support Windows.

3. Configuration

Generate a base configuration file: echo_supervisord_conf > /etc/supervisord.conf Key sections include [unix_http_server], [supervisord], and [supervisorctl], with parameters for socket files, logging, and process definitions.

Custom service configuration files can be included via the [include] directive, allowing modular management of multiple projects.

Application Configuration Example

[program:test]
command=python -u ./test.py
directory=/root/test/
redirect_stderr=true
stdout_logfile=/root/test/test.log

Link the application config into the include directory:

ln ./supervisor-test.ini /etc/supervisord.d/supervisor-test.ini

4. Running Supervisor

Start the daemon: supervisord Control processes with supervisorctl (e.g., status, start, stop, restart, signal).

5. Advanced Features

Process Group Management

[group:test]
programs=test-task_service, test-collector

Commands can target groups using the syntax groupname:.

Program Parameter Details

command

: command to run. directory: working directory. numprocs and process_name: run multiple instances. autostart: start on daemon launch. stdout_logfile, stdout_logfile_maxbytes, stdout_logfile_backups: log handling. redirect_stderr: merge stderr into stdout.

Supervisorctl Commands

Supported commands include add, remove, status, start, stop, restart, signal, update, etc.

Signal Handling in Applications

import time, signal
RUN = True
def exit_handler(signum, frame):
    print(f'processing signal({signal.Signals(signum).name})')
    print('update task status')
    print('clear cache data')
    global RUN
    RUN = False
signal.signal(signal.SIGTERM, exit_handler)
while RUN:
    print(time.strftime('%Y-%m-%d %H:%M:%S'))
    time.sleep(1)
print('exited')

Supervisor can send signals to managed processes (e.g., signal 15 test) to trigger graceful shutdown logic.

Web Interface

[inet_http_server]
port=0.0.0.0:9001
username=user
password=123

After enabling and restarting, the web UI provides visual process control (ensure it is not exposed publicly).

6. Automatic Restart Mechanism

The autorestart option controls restart behavior ( true, false, unexpected), while exitcodes defines expected exit codes. startsecs determines the minimum runtime for a process to be considered successfully started, and startretries limits restart attempts.

7. Summary

Supervisor offers a robust solution for managing persistent services, with flexible configuration, group management, signal handling, and both CLI and web interfaces, making it suitable for DevOps and operations workflows.