[custom-header-login]
Reverse Path Filtering (rp_filter) and Martians (log_martians) for LPIC-3 Security
The IPv4 setting for rp_filter or Reverse Path filtering is a method used by the Linux Kernel to help prevent attacks used by Spoofing IP Addresses, yes the Internet is not a safe place and people aren’t always whom they say they are. Reverse path filtering is a Kernel feature that, when enabled, is designed to ensure packets that are not routable to be dropped. The easiest example of this would be and IP Address of the range 10.0.0.0/8, a private IP Address, being received on the Internet facing interface of the router. These packets can’t be routed to the Internet and should not be there, if they are something bad is happening. By default the setting is enabled, relax you can breathe again. With the setting disabled a router will forward any packet not matter if the source address is doubtful, not a great setting to have, have your medication ready again. The enabled by default is Linux being the grown-up and keeping you safe and the idea is to limit Denial of Service attacks where many spoofed addresses will hit a server. If the source addresses are not routable back, the server will be unable to complete TCP synchronization and connections will be held open waiting for timeouts. Just as in a supermarket when all of the cashiers are busy no-one gets servers. On the Linux server when all of the connections are held open trying to synchronize with devices they don’t exist, no new connections can be established and the bad guys have won. IPv4 rp_filtering is a good thing and we shall investigate the options and how we can implement IP Address Spoofing in Linux. We shall see how the different settings to rp_filter effect the routing of the spoofed packets.
First I want you to be able to understand some common settings within the procfs and comprehend the incomprehensible regular expressions a little better. Let’s take a look at the extended regular expressions that we can use with the options -r or –pattern option to sysctl:
$ sysctl -ar 'rp_filter' net.ipv4.conf.all.arp_filter = 0 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.arp_filter = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eth0.arp_filter = 0 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.lo.arp_filter = 0 net.ipv4.conf.lo.rp_filter = 0
In this first example, we use the pattern ‘rp_filter‘ . This expression returns any line that contains a match for the string supplied. This includes arp_filter which we do not want to display at this time, along with the matching rp_filter lines we do want. We might think that we can extend the filter to be ‘.rp_filter‘ as the dot precedes the text we desire and excludes arp_filter. This will NOT work as expected because, within a regular expression, the dot represents ANY character and as so , would match on both a literal . and the a from arp.
$ sysctl -ar '.rp_filter' net.ipv4.conf.all.arp_filter = 0 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.arp_filter = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eth0.arp_filter = 0 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.lo.arp_filter = 0 net.ipv4.conf.lo.rp_filter = 0
As we can see we still have the same results set, as we predicted. To apply a literal meaning to our . we will need to escape the dot within the regular expression removing any special meaning.
$ sysctl -ar '\.rp_filter' net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eth0.rp_filter = 1 net.ipv4.conf.lo.rp_filter = 0
We now see in the results set, that we have successfully displayed the configuration for rp_filter.
The next challenge is to understand the configurations from the subtrees: all, default, eth0 and lo etc.
The default subtree affects all new interfaces. Assigning a value to a configuration key beneath default will not affect existing interfaces but acts as a template for new interfaces.
As of Kernel version 2.6.31 the highest value set in either the interface specific subtree or the all subtree will be effective setting. If we needed to disable rp_filtering, which uses a value of 0, then we would need to do this in the both in the interface specific key and the all key to ensure that there is not a higher value in any configuration key effecting the interface.
The highest value becomes the effective value!
The 3 values that can be set for the key rp_filter are:
0 No source address validation is performed and any packet is forwarded to the destination network
1 Strict Mode as defined in RFC 3704. Each incoming packet to a router is tested against the routing table and if the interface that the packet is received on is not the best return path for the packet then the packet is dropped.
2 Loose mode as defines in RFC 3704 Loose Reverse Path. Each incoming packet is tested against the route table and the packet is dropped if the source address is not routable through any interface. The allows for asymmetric routing where the return path may not be the same as the source path
The default value we can see is set to 1 in all cases except the loop-back interface: lo.
To be able to test and demonstrate these modes we will need 3 systems. The demo environment uses virtual machines and we make use of two host-only networks 192.168.58.0/24 and 192.168.59.0/24. The connectivity to the Internet is through the bridged interface on the router host.
Host net2:
NIC: enp0s3
IP Address: 192.168.58.4/24
Default route: 192.168.58.3
Host router:
NIC: enp0s3
IP Address: 192.168.0.3/24 Default route to Internet
NIC: enp0s8
IP Address: 192.168.58.3/24
NIC: enp0s9
IP Address: 192.168.59.4/24
Host net3:
NIC: enp0s8
IP Address: 192.168.59.3/24
Default route: 192.168.59.4
Testing of rp_filter we make use of the command ping and ICMP packets are allowed to all hosts. The router provides the routing of packets between net3 and net2 and it is the router host that will receive all the configuration changes using sysctl. On the host net2 we will implement IP Address Spoofing.
Amazingly, as we have called the router host “router” we need to enable IPv4 routing on the this system, this is still managed with sysctl:
router$ sudo sysctl -w 'net.ipv4.ip_forward=1'
With this enabled you should be able to test your connectivity and ping host net3 by its IP Address from host net2 and vice-versa.
Now we will point on out black-hat and start spoofing IP Addresses from host net2 to host net3. On host net2 we set up the spoofing using the command iptables, it is much easier than you think:
If you do not SSH to your host then the first line does not need to be added. During the demo I need to SSH to the net2 host. Adding the first line ensures that we do spoof the SSH source address.
net2$ sudo iptables -t nat -A POSTROUTING -p tcp -m tcp --dport 22 -j ACCEPT net2$ sudo iptables -t nat -A POSTROUTING -j SNAT --to-source 192.168.59.10
Any non SSH packet leaving the host net2 will now have the address of 192.168.59.10, don’t expect to receive and packets back to the source host, net2. Remember we are spoofing and attempting a DoS attack and the idea is that there is no route back to the source host.
The address that we spoof is from the net3 network that is not in use. The router will not expect this packet to originate from the NIC facing the 192.168.58.0/24 network. Moreover, the router will know that this network is accessible via a different NIC and the default rp_filter settings will drop the packet. Hopefully from a great height, in any case it will not be forwarded to the target host and no DoS attack will be implemented.
Where rp_filter has the default value of 1 and Strict Mode enabled the packet will not be forwarded and dropped by the router.
We will test this by running the packet capture program tcpdump of both the router and net3. The packet will be received on the router but NOT host net3. :
router$ sudo tcpdump -i enp0s8 -p icmp net3$ sudo tcpdump -i enp0s3 -p icmp
If your NIC names are different use the correct NIC name for your system. On the router it should be the interface facing network2. On the net3 host the NIC should be the only interface connected to network3. The host net3 will be the PING target and the router will need to route the packets to the correct network.
From host net2 we will try the ping:
net2$ ping -c1 192.168.59.3 PING 192.168.58.3 (192.168.59.3) 56(84) bytes of data. --- 192.168.59.3 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0
The ping will fail as we are using a spoofed address.
If we check on the host net3, the ping will not have arrived:
Host net3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
The tcpdump screen is as before with no collected data.
On the router we will see the collected data:
Host router
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes 18:23:07.472986 IP 192.168.59.10 > 192.168.59.3: ICMP echo request, id 1604, seq 1, length 64
We see the single ICMP packet that we sent from the spoofed address of 192.168.59.10 to the destination 192.168.59.3. The default setting of rp_filter has protected the network, we will see later how we can read this is the log files.
We can leave tcpdump running on the host net3 but we will use CTRL+C to close it on the router host. On the router we will look at configuring other modes for rp_filter.
We will now configure the router with the so called loose mode of rp_filter. Setting this will require the value of 2 to be configured. In doing so we will allow asymmetric routing. This allows packets to pass so long at the destination network is routable even if it is not routable using the same interface. We will notice that the ICMP packet is received on the net3 host as it will not be dropped by the router. We will still not see any ping replies on the originating clients as it cannot receive an returned packets as the spoofed address is on another network.
Asymmetric routing can be used as a method of load balance when the route from a network will be different from the route to a network. Similar to one-way systems in crowded cities.
Configuring a value of 2 in the rp_filter key will only necessitate the configuration with in the all subtree. It is the highest value that will be effective and 2 is higher than the interface specific key.
On the router host we issue the following command:
router$ sudo sysctl -w net.ipv4.conf.all.rp_filter=2 net.ipv4.conf.all.rp_filter = 2
With the tcpdump command still running on target host net3. We will ping again from host net2:
$ ping -c1 192.168.59.3 PING 192.168.58.3 (192.168.59.3) 56(84) bytes of data. --- 192.168.59.3 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms
By viewing the console on host net3 we will see that packet arrives in tcpdump:
Host net3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes 08:39:32.860631 IP 192.168.59.10 > 192.168.59.3: ICMP echo request, id 2685, seq 1, length 64
The packet arrives on the target host as the router had a route defined to the source network, even though it was through a different NIC.
Disabling rp_filter all together is not recommended and I can’t really think of reasons why you would want to. As we would use a value of 0 to disable the reverse path check this will need to be set in both the interface specific key and the all key. Remember, the highest value is the effective value so 0 would always be lower the 1 or 2. On the demo system it is NIC enp0s8 that faces network 2 and it is this NIC that will need the have the setting disabled.
router$ sudo sysctl -w net.ipv4.conf.all.rp_filter=0 net.ipv4.conf.all.rp_filter = 0 router$ sudo sysctl -w net.ipv4.conf.enp0s8.rp_filter=0 net.ipv4.conf.enp0s8.rp_filter = 0
With the setting disabled we will no longer check the reverse path of the source address from interface enp0s8 of the router. We will still be able to ping the host net3 as in the previous demonstration but no reverse check will be performed.
We now want to be able to look at the logs that record the dropped packets by rp_filter, for this we will need to setting enabled in Strict or Loose mode again. We will return both keys to their default value before look at logging.
router$ sudo sysctl -w net.ipv4.conf.all.rp_filter=1 net.ipv4.conf.all.rp_filter = 0 router$ sudo sysctl -w net.ipv4.conf.enp0s8.rp_filter=1 net.ipv4.conf.enp0s8.rp_filter = 0
Mis-configured hosts and even IoT devices including printers may cause these packets to be dropped. You are not always under attack when these events happen. But knowing what is happening is always useful. Even if it is a mis-configured device being able to identify the issue in the logs will help. The key log_martians is what we look at now. The idea of Martian packets is that they should not be there and perhaps they are from Mars the planet. Logging is turned of by default but we can enable this is we want a view on these dropped packets on the router. They will log to the kernel log file, on CentOS /var/log/message will shows this, on Ubuntu it is /var/log/syslog.
On the router we enable logging:
router$ sudo sysctl -w 'net.ipv4.conf.all.log_martians=1' net.ipv4.conf.all.log_martians = 1
With the setting enabled we will follow the log file on the router
router$ sudo tail -fn0 /var/log/syslog
Now we will ping the net3 host again from net2. Whist pinging the host net3 from host net2 the log files displays the long awaited Martians:
May 2 11:21:19 router kernel: [62450.100636] IPv4: martian source 192.168.59.3 from 192.168.59.10, on dev enp0s8
The Martians have landed and they are in our log file right now! Searching the log file for the text Martians should give you the indication that you need that you have mis-configured hosts or possible DoS attacks on your system.
We should now return host net2 to its normal IP Address and to stop spoofing:
net2 $ sudo iptables -t nat -F
The video follows