This article was posted here
Most of the time we use iptables to set up a firewall on a machine, but iptables also provides packet and byte counters. Every time an iptables rule is matched by incoming or outgoing data streams, the software tracks the number of packets and the amount of data that passes through the rules.
It is easy to make use of this feature and create a number of "pass-through rules" in the firewall. These rules do not block or reroute any data, but rather keep track of the amount of data passing through the machine. By using this feature, we can build a simple, effective bandwidth monitoring system that does not require additional software.
For a desktop computer, you may need to create only two rules to log the total input and output. A system acting as a router could be set up with additional rules to show the totals for one or more subnets, right down to the individual IP address within each subnet. In addition to knowing exactly how much bandwidth each host and subnet on the network is using, this system could be used for billing or chargeback purposes as well.
Rules setup
The rules setup itself is quick and straightforward, and takes only a few minutes. Obviously, you need to be root or use sudo to insert iptables rules.
The examples in this article are based on a router that provides Internet service to various towns. The iptables rules keep track of how much bandwidth each town uses and how much bandwidth each customer in that town uses. At the end of each month, an administrator checks the counters. Individuals who use more than they were supposed to get billed for over usage, the counters are reset to zero, and the process is repeated at the beginning of the next month.
The IP addresses in this article are modified from the real addresses. We'll use the private IP space 192.168.0.0/16, subnetted into smaller blocks.
First, we will create two custom chains for the two towns and put town-specific rules in them. This will keep the built-in FORWARD chain relatively clean and easy to read. In this example, the FORWARD chain will only provide the global counters (all customers combined on a per-town basis).
iptables -N town-a
iptables -N town-b
The next data element is the total bandwidth counter. Because this machine is a router only, the INPUT and OUTPUT chains are of little interest. This machine will not be generating a significant amount of bandwidth (i.e., it is not serving as a mail or Web server), nor will it be receiving significant uploads from other hosts.
Total bandwidth downloaded by and uploaded to the two towns combined:
iptables -A FORWARD
This is the easiest of rules. The rule will match any source and any destination. Everything that is being passed through this router matches this rule and will provide the total of combined downloaded and uploaded data.
We also want to see how much each town downloads and uploads separately:
# Town A Downloads
iptables -A FORWARD -d 192.168.1.0/26 -j town-a
# Town A Uploads
iptables -A FORWARD -s 192.168.1.0/26 -j town-a
# Town B Downloads
iptables -A FORWARD -d 192.168.1.64/27 -j town-b
# Town B Uploads
iptables -A FORWARD -s 192.168.1.64/27 -j town-b
The use of source and destination in the above rules may be a source of confusion. Destinations are often equated with uploads, and sources are downloads. This would be true whether the data was destined for the router or originated from the router itself.
In this application, however, we reverse the perspective. This router is forwarding (uploading) data to a destination, but from a customer perspective, data is being received. In other words, the customer is downloading that data. When dealing with customers, the terminology is data they downloaded, not what the router uploaded to them. This is why in the FORWARD chain, the terms destination and source typically have reversed meanings.
The rules created above give us separate totals for all downloads to and uploads from each individual town. This is accomplished by matching the source and destination of all traffic through the router for a town's specific subnet. After a rule is matched, the -j option invokes a jump to one of the custom chains. These custom chains can then be used to add additional rules pertaining to the subnet. For instance, rules can be created for each individual IP address in that subnet to track bandwidth on a per-host basis:
# Town A, Host 192.168.1.10 Download
iptables -A town-a -d 192.168.1.10
# Town A, Host 192.168.1.10 Upload
iptables -A town-a -s 192.168.1.10
You could repeat this process for every IP address for all towns within the subnet.
Bandwidth statistics
Viewing the current bandwidth usage is a matter of running iptables with the -L and -v options. The -L outputs the statistics for a chain (or all chains if none is provided). The -v option provides verbose output, including the packet and byte counters that we are interested in. I recommend using the -n option as well to prevent DNS lookups, meaning iptables will show the IP addresses without attempting to resolve the hostnames for the IP addresses, which would put additional and unnecessary load on the router.
The output below is modified from the full output for brevity:
root@raptor:~# iptables -L -v -n
Chain FORWARD (policy ACCEPT 7936M packets, 3647G bytes)
| bytes | target | source | destination |
| 338G | 0.0.0.0/0 | 0.0.0.0/0 |
| 104G | town-a | 0.0.0.0/0 | 192.168.1.0/26 |
| 40G | town-a | 192.168.1.0/26 | 0.0.0.0/0 |
| 20G | town-b | 0.0.0.0/0 | 192.168.1.64/27 |
| 12G | town-b | 192.168.1.64/27 | 0.0.0.0/0 |
This snippet shows that towns A and B combined have downloaded and uploaded a total of 338GB. Town A is responsible for 104GB downloaded and 40GB uploaded. In the first line of output of the chain itself is a "more" total number -- 3,647GB. This is the total amount of data routed through since the last time this router was restarted, or more accurately, since the last time the iptables modules were inserted into the kernel.
When a chain is "zeroed" (resetting all counters in a chain to zero) with the -Z option, this number is not reset. For this reason, I recommend creating a real total rule to make it easier to reset the total counter. It then takes one command to reset the counters, and you do not need to remove modules, restart the server, or work with the iptables-save and iptables-restore commands to reset the counter.
Scrolling further down the output shows the individual IP addresses. Example for Town A:
Chain town-a (2 references)
| bytes | source | destination |
| 32G | 0.0.0.0/0 | 192.168.1.10 |
| 282M | 192.168.1.10 | 0.0.0.0/0 |
| 1521M | 0.0.0.0/0 | 192.168.1.11 |
| 656M | 192.168.1.11 | 0.0.0.0/0 |
This output further breaks down the total bandwidth of Town A down to the individual customers.
The "2 references" shown in the iptables output refer to the two rules in the FORWARD chain that jump to this chain.
Saving data across reboots
If you reboot the machine or remove the iptables kernel modules, you'll lose all of your packet and byte counters. if these counters are to be used for billing purposes, you will want to make backups of the running counters, and in the event of a reboot, restore the counters rather than starting from zero.
The iptables package comes with two programs that aid in this: iptables-save and iptables-restore. Both programs need to be told to explicitly use the packet and byte counters during backup and restore using the -c command line option.
The backup and restore process is fairly straightforward. To back up your iptables data, use iptables-save -c > iptables-backup.txt. To restore the data, after reboot, use iptables-restore -c <. iptables-backup.txt
Conclusion
Iptables provides a quick and easy way to track bandwidth usage without having to install additional software. You have, and probably already use, the tools needed to accomplish this monitoring.
The flexibility and power of iptables allows for more complex monitoring scenarios. You can create rules to not only track different subnets, but also to track specific ports and protocols, which lets you track exactly how much of each customer's traffic is Web, email, file sharing, etc.
In addition, these bandwidth monitoring rules can also become blocking rules. If a host has used too much bandwidth, its rule in a town's specific chain can be modified by adding -j DROP to both the download and upload rules. This effectively stops traffic being routed to and from that host.
