Shell Scripts for Automatically Counting Failed IP Addresses and IPs Within a Ten‑Minute Window

Recently a colleague asked the author about writing shell scripts for log analysis, prompting a detailed walkthrough of two practical problems often encountered in production environments.

Problem 1: Automatically count the IP addresses that caused failed website accesses. Problem 2: Automatically count the IP addresses that accessed the website within the last ten minutes.

For the first problem the author defines a failure as an HTTP status code such as 400, 403, 404, or 500, which can be identified in the access log. The IP address can then be extracted with awk '{print $1}' , sorted and counted.

For the second problem the log entries contain timestamps, so the script must filter lines whose fourth field falls between the current time and ten minutes ago. This is achieved by using the date command to generate the two timestamps and an awk expression to select the appropriate range.

Script for counting failed IPs (fail_connect_ip.sh):

########################################## # this script is for auto check client connect to webserver failed # # created by mingongge on 2017-01-01 # ########################################## #!/bin/sh egrep "400|403|404|500" /wwwlogs/access.log | awk '{print $1}' | sort -nr | uniq -c

Running the script yields a list of IPs with the number of failed requests, e.g.:

[root@ ~]# sh fail_connect_ip.sh 1 29.20.20.3 1 2.11.22.1 8 1.1.1.1

Script for counting IPs in the last ten minutes (client_connect_ip.sh):

########################################## # this script is for auto check client IPs # # created by mingongge on 2017-01-01 # ########################################## #!/bin/sh TIME=$(date +%d/%b/%Y:%k:%M:%S) NTIME=$(date +%d/%b/%Y:%k:%M:%S -d '-10 minutes') # define time variables awk '$4 >= "['$NTIME']" && $4 <= "['$TIME']"' /wwwlogs/access.log > log.txt # get log entries within the time window awk '{print $1}' log.txt | sort -nr | uniq # extract, sort and deduplicate IP addresses

The article also shows how to obtain the current time and the time ten minutes ago using date +%d/%b/%Y:%k:%M:%S and date +%d/%b/%Y:%k:%M:%S -d '-10 minutes' , respectively.

In summary, the author demonstrates a straightforward approach: break down the requirement, use standard Unix utilities (awk, date, sort, uniq), and combine them into small scripts, illustrating that a clear logical flow leads to effective solutions.