Using Fail2ban to ban IP by Apache / Nginx response code

Some of our old CMS systems are returning 403 and 404 codes when the site is attacked with various payloads. The application firewall responds on a deeper level (the application code has to be executed) so the speed of such solution is always worse than rejecting connection on the iptables level.

One of the common software used to block remote hosts is Fail2ban. It is available for almost every Linux distribution. I will not get into details on how to install it.

By default, Fail2ban is installed with the sample set of rules and filters. Since there is no ready-made filter for Apache or Nginx response codes, we have to create one.

Let’s start with the filter definition:

#/etc/fail2ban/filter.d/code-40X.conf
[Definition]
failregex = ^<HOST> - .* "(GET|POST|HEAD).*HTTP.*" 40(3|4) .*$
ignoreregex =

The above filer is looking for all log entries with 404/403 code. Of course, you can adjust it to match your preferences. Once the filter definition is ready, it is time to adjust the jail configuration. The jail configuration should be handled in the “/etc/fail2ban/jail.local” file. The “/etc/fail2ban/jail.conf” should be left as is.

At the end of the “/etc/fail2ban/jail.local” file we can now add:

[code-40X]
enabled = true
port = 80,443
protocol = tcp
filter = code-40X
maxretry = 5
bantime = 1800
logpath = /var/log/apache2/*access.log

In the example above, you may want to adjust the last three lines. The “max retry” dictates the number of log entries for given IP to ban the IP, the “ban time” is the time in seconds during which the IP will be banned. The “log path” is the actual path to the log files, which can also contain wildcard characters if you are using different log files for different sites.

Once the files are adjusted, you can restart Fail2ban and perform tests.

Whitelisting

In our case we had a CDN configured to provide images for the website. Since the above fail2ban filter may easily ban such CDN in case of missing images, we added the “ignore regex” statement to the filter. The filter file looks like this now:

[Definition]
failregex = ^<HOST> - .* "(GET|POST|HEAD).*HTTP.*" 40(3|4) .*$
ignoreregex = ^<HOST> .* "OUR CDN USERAGENT"$

As you can see, our CDN is using a particular user agent string, which we used to ignore requests performed by the CDN.