Fail2ban routing actions

I said I should be doing some more technical posts so here we go! Blinkenshell runs the SSH server on a non-standard port (mainly port 2222, but also port 443 for people trying to avoid some firewalls), and the reason for doing this is to avoid some automated bots that go around the internet scanning for open SSH servers and trying different bruce-force attacks to log in. I still think this non-standard port helps a bit, but there’s always some more persistent attackers out there that find SSH servers running on other ports and start hammering away there as well, and for this reason we have fail2ban. Fail2ban will scan through any logfiles you specify and apply certain filters/patterns to find failed login attempts, and if there are repeated failed attempts it will try to perform different actions like blocking the source IP using iptables. For Blinkenshell I try to avoid running too many programs on the main SSH server triton because of security reasons, so our setup is a little bit different.

The first part of the puzzle is rsyslog running on triton sending syslog messages over UDP to a separate log collection server. This is useful for several reasons like trying to figure out why a server crashed or to get forensic data after a host was compromised. In this case we’re going to use it to be able to run fail2ban on a separate host from the one we’re trying to protect. So we install fail2ban on the log collection server and set up a new “jail” that will listen for logs coming from triton. Something like this:

[sshd-triton]
port = ssh,2222
logpath = /var/log/remote/triton.log
enabled = true
filter = sshd[mode=aggressive]
banaction = route

The second part of the puzzle is hinted at the last line “banaction = route” here. Since fail2ban will not run on the same server as the server we’re trying to protect any iptables rules that we install locally on the fail2ban server will not stop the attackers. The idea here is to add any banned IPs to the local routing table, and then send these routes to the firewall and drop the traffic there. This will require a routing protocol daemon running on the fail2ban server that talks to a routing daemon on the firewall.

In my case the fail2ban server runs on Linux so here I’ll choose FRRouting (FRR) for the routing daemon. My preferred choice of routing protocol for cases where you want to specify some routing policy is BGP, so I’ll enable the bgpd in /etc/frr/daemons and systemctl restart frr. You can then enter a cisco-style cli using the command “vtysh”, and go into configure mode using “configure”. Sample config:

ip prefix-list BLACKHOLE seq 5 deny 10.0.0.0/8 le 32
ip prefix-list BLACKHOLE seq 10 deny 194.14.45.0/24 le 32
ip prefix-list BLACKHOLE seq 15 permit 0.0.0.0/0 ge 32
!
route-map BLACKHOLE permit 10
 match ip address prefix-list BLACKHOLE
!
router bgp <myasn>
 neighbor <firewallip> remote-as <firewallas>
 !
  address-family ipv4 unicast
   redistribute kernel route-map BLACKHOLE

To try and avoid any accidental dropping of legitimate traffic I’ll add a little route-map to deny my local prefixes first, and then allow any /32 routes. The “redistribute kernel” line is what will actually take the routes that fail2ban added to the kernel routing table using “banaction = route” and add them to the BGP table.

On the firewall end I’m using OpenBSD so there we will be using openbgd instead of FRR, so it’s a completely different syntax! I’m actually already running bgpd for other things on the firewall (maybe more updates on this later!), but the relevant parts to this fail2ban blackhole thing are:

prefix-set accept-blackhole-in {
    0.0.0.0/0 prefixlen = 32
}
neighbor <fail2ban ip> {
    remote-as <fail2ban as>
    descr "fail2ban"
}

allow from <fail2ban ip> prefix-set accept-blackhole-in set pftable "blackhole"

Again, another filter to only accept /32 routes so we don’t ruin other routing by some misconfiguration. The other key part here is: set pftable “blackhole”. This will add any routes received from this neighbor to a table in pf. We can then refer to this table when writing firewall rules in pf like so:

table <blackhole> persist
block in quick log on $if from <blackhole> to any

Or if you want you can re-route the traffic to some honeypot etc with “route-to” options in pf. This will block all traffic from the banned IP in the firewall, so that IP address will not be able to reach any other services hosted behind the firewall either. This could be seen as extra protection, but it could also cause even more confusion for users that accidentaly type the wrong password too many times and then can suddenly not reach any other services either 🙂 We’ll see how it goes!

Let me know if you’re interested in more technical stuff like this