Why Does rsyslog Fail Inside Docker? Uncovering Common Logging Pitfalls
This article walks through a real‑world Docker logging issue, detailing how syslog and rsyslog behave inside a CentOS 7 container, the failed attempts to fix it, and the eventual discovery of missing /dev/log socket and permission constraints.
Background
The author’s team built a Docker + K8s container platform and encountered a problem: when starting rsyslog inside a CentOS 7.2 container, log messages never appeared in /var/log/messages . Initial checks of rsyslog.conf showed no obvious misconfiguration.
Investigation Process
Idea 1 – Try Privileged Mode
Running the container with --privileged did not solve the issue. Research revealed that systemd inside containers requires the CAP_SYS_ADMIN capability, access to the cgroups filesystem, and removal of certain unit files.
systemd requires CAP_SYS_ADMIN (hence --privileged is not ideal for a base image).
systemd needs access to the cgroups filesystem.
Several unit files cause errors if not removed.
Even with --privileged , systemd could not start, leading to the suggestion to launch the container with /sbin/init as the entrypoint:
docker run --privileged -itd centos:latest /sbin/initIdea 2 – Use an Official rsyslog Image
The author searched Docker Hub for an rsyslog image and pulled quiver/rsyslog . The container started successfully and rsyslog worked, prompting a version comparison:
Official image: rsyslog-7.6.7-1.el7.x86_64
Locally built image: rsyslog-7.4.7-12.el7.x86_64
Upgrading to the newer version resolved the logging problem.
Root Cause Analysis
Further inspection of the rsyslog configuration revealed the $ModLoad imuxsock module, which reads logs from the Unix socket /dev/log . In the failing container, the socket was missing:
netstat -anlp | grep /dev/logWhen the container was started with /sbin/init , systemd created the socket, and rsyslog could write logs. The missing socket in the original setup was due to Docker’s permission restrictions and a bug in rsyslog’s socket creation logic.
Conclusion
The troubleshooting journey shows that Docker logging issues often stem from container privileges, missing systemd‑created sockets, and version mismatches. By examining Docker’s constraints, trying privileged mode, and ultimately using a compatible rsyslog image, the author identified and resolved the root cause.
360 Zhihui Cloud Developer
360 Zhihui Cloud is an enterprise open service platform that aims to "aggregate data value and empower an intelligent future," leveraging 360's extensive product and technology resources to deliver platform services to customers.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.