How to Efficiently Audit Linux System Operations: Tools, Tips, and Filtering Rules

Detailed recording of host system actions (e.g., user creation, file renames, command execution) aids security audits and fault tracing, but excessive logs increase analysis cost, especially when aggregated from many hosts.

Most system activities are repetitive and unnecessary for auditing, such as cron jobs or trusted daemons, so filtering rules should be applied to ignore redundant or sensitive entries.

Ignore records generated by cron or daemon processes.

Ignore command lines or scripts that contain passwords.

Ignore logs from monitoring users (e.g., nagios, zabbix, prometheus).

Ignore high‑frequency logging operations.

When transmitting logs in plain text to a remote server, it is advisable to skip the second rule. For network‑related syscalls like

connect

and

accept

, logging every occurrence can overwhelm the kernel and network, so enable such logging only when troubleshooting.

History Recording Method

The traditional history approach simply forwards command history to syslog, but it suffers from several drawbacks:

Easy to modify or bypass.

Lacks context information (pid, uid, sid, etc.).

Cannot capture operations inside shell scripts.

Cannot record non‑login activities.

Difficult to implement filtering rules.

Custom Bash Recording Method

Customizing the Bash source to emit audit logs adds some context but shares many limitations with the history method, including easy bypass (users can switch shells), inability to log script‑internal actions, limited filtering, and the maintenance burden of updating Bash versions.

Snoopy Recording Method

Snoopy wraps

execv

/

execve

via a preload library, capturing every command execution with full process context. Its advantages are:

Hard to bypass once

PRELOAD

is set.

Records commands regardless of a tty session.

Captures operations inside shell scripts.

Logs command arguments (including usernames and passwords).

Rich filtering rules (by daemon, uid, etc.).

Example log entry:

<code>Oct 27 11:34:31 cz-t1 snoopy[24814]: [time_ms:778 login:cz uid:0 pid:24814 ppid:24676 sid:24579 tty:/dev/pts/0 cwd:/root filename:/bin/uptime username:root]: uptime -p</code>

This shows the

uptime -p

command executed by root with full context.

Only supports

execv

/

execve

syscalls.

Without proper rules, log volume can become large.

Currently lacks built‑in sensitive‑information filtering.

Typical snoopy filter configuration to ignore cron, a custom daemon, and the zabbix user:

<code># zabbix uid is 992
filter_chain = exclude_uid:992;exclude_spawns_of:crond,my-daemon</code>

The filtering logic resides in filtering.c (function snoopy_filtering_check_chain ) and is applied at log‑generation time, not beforehand.

Additional

exclude_comm

rule can ignore specific commands (e.g., mysql, mongo, redis‑cli) to prevent password leakage:

<code>filter_chain = exclude_uid:992;exclude_comm:mysql,mongo,redis-cli</code>

Auditd Recording Method

auditd leverages kernel‑level support (kauditd) to provide a comprehensive framework capable of monitoring virtually any event, surpassing snoopy’s limited syscall coverage.

Sample audit log with extensive context:

<code>type=SYSCALL msg=audit(1603800704.305:5304075): arch=c000003e syscall=59 success=yes exit=0 a0=1c79fd0 a1=1bf51a0 a2=1bd4450 a3=7ffe7270d320 items=2 ppid=95264 pid=99702 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=571973 comm="mysql" exe="/usr/bin/mysql" key="command"
type=EXECVE msg=audit(1603800704.305:5304075): argc=5 a0="/usr/bin/mysql" a1="-h" a2="127.0.0.1" a3="-P" a4="3301"</code>

Auditd’s architecture separates policy configuration (

auditctl

) from the daemon (

auditd

) and its dispatcher (

audisp

). Policies are defined via

-a

or

-w

rules, typically stored in

/etc/audit/rules.d/audit.rules

. Example rules to ignore common tools and monitor kernel module loading:

<code>### ignore common tools
-a never,exit -F arch=b64 -F exe=/usr/bin/redis-cli
-a never,exit -F arch=b64 -F exe=/usr/bin/mysql
-a never,exit -F arch=b64 -F exe=/usr/bin/mongo

# Kernel module loading and unloading
-a always,exit -F perm=x -F auid!=-1 -F path=/sbin/insmod -k modules</code>

Note: never rules support filtering by exe , while always does not, making exclude_comm in snoopy more convenient for command‑based exclusions.

eBPF Recording Method

eBPF, available in Linux 4.1+, enables dynamic tracing. Tools like

bpftrace

and

bcc

(e.g.,

execsnoop

) can capture all

execv

/

execve

executions:

<code># ./execsnoop
PCOMM            PID    PPID   RET ARGS
bash             32647  32302    0 /bin/bash
id               32649  32648    0 /usr/bin/id -un
hostname         32651  32650    0 /usr/bin/hostname
uptime           410    32744    0 /bin/uptime</code>

eBPF requires Linux 4.1+ (full feature set from 4.10). Distributions such as CentOS 8 or Debian 10 are recommended; older kernels (e.g., CentOS 7.6) have limited support.

Conclusion

Auditing Linux system operations simplifies problem tracing and can serve as forensic evidence; security‑sensitive enterprises often adopt auditd for comprehensive auditing. In practice, using snoopy or auditd together covers most needs, while eBPF provides fine‑grained tracing. Sending logs to platforms like ELK is useful, but careful filter design is essential to avoid overwhelming data volumes.