Introduction
Have you ever wondered what was happening on the front lines of your LAN fortification device, but you were put off by the ugly firewall logs that seem to both have too much information, while also seem to be lacking in some very important information at the same time?
Maybe at some point you wanted to get a general idea of who was trying to access your network, so you thought of enabling some of the DNS lookup features available in your various server software logs? I thought of this, but upon doing some research on mailing-list archives, the typical answer seemed to be "just don't do it, it takes up too many resources, and it's not worth it". Maybe you really wanted to be doing reverse DNS lookups on every IP address that accesses your network, but when thinking about the ramifications you realized that it would generate way too much extra traffic on your connection, and you came to the conclusion that it's just not feasible.
Well the answer "just don't do it" wasn't good enough for me, so I wrote a very simple Bourne Shell script that tackles this problem easily. The script which accompanies this tutorial addresses all these problems which I have had personally, and I'm sure if you're here, you have had these problems too.
This will teach you how to take your cryptic iptables firewall log from your routing device, and use it to generate a more meaningful log (in near-real-time) that has extra information and is much easier to read. Some of the info from the original log is not added to the new log at all, such as port numbers, packet length, etc.
but other more important information is added that you might need in case of a break-in.
I have written a script to make this all happen smoothly and you can have it for free. This script will maintain a flat-file of all external IP addresses
that try to touch your network, and all IP addresses that your network contacts on the outside. A separate log file will be written containing a timestamp of the first time your network and a new IP touch. The log will also contain the IP, its reverse DNS address, and the direction of the network touch.
The script will use this "IP database" file to make sure DNS lookups are only performed the first time any IP address meets with your network. If this is undesirable, or if you want a fresh database or log file after a certain period of time, you need only rename the database file and/or the log file, and the script will gracefully continue, immediately creating new database and/or log files and continuing operations on the new files as normal. You can even automate this renaming process very easily with a cron job, but I'm not going to tell you how because it's really easy and you can figure it out for yourself.
Please note that doing a DNS lookup generates extra traffic on your network, and across the internet. The script, by design, only logs and resolves addresses the first time they touch your network. As such, the log generated by this particular script is not a general-purpose catch-all log with all the info you need for everything.
Subsequent touches beyond the first for any logged IP address are not logged at all, but if you want all touches logged, I urge you to modify the script and implement it as an optional feature, it would be trivial and I offer it as a simple challenge to the reader (because it's educational and fun and I'm lazy).
Because doing too many DNS lookups is hard on the internet backbone and too much traffic too quickly could bloat your log file and database, this script will occasionally discard new IPs, not logging them and not doing DNS lookups, but only to prevent too much damage from being caused by an advanced DDoS attack, and only when too many new IPs are coming in too quickly.
The logging hit rate is currently tuned for low to moderate traffic, but it can be hack-tuned by adjusting the sleep values in the script. You will probably want to test this a bit if you want high-precision logs on high-traffic sites. Higher precision (lower sleep values) means higher CPU and I/O load, and it means you will need more disk space as the log and IP database will be bigger.
The official version of my script has only been tested on the iptables firewall log from a Tomato firmware powered router (no I will not tell you which router or which version of iptables). If you know any shell scripting I urge you to modify the script to make it work with other routers and other types of firewalls. If you had to modify it to get it to work, please post a link to your updated version in the comments section below.
Procedure
I recommend setting up your router to log onto a separate Linux or *nix box on your LAN. You can find lots of info on how to do this with a simple online search. If you don't want to do this, you at least need a Linux or *nix-based router, with shell access, and it needs to have some standard commands available on it. But just set it up to log onto another network box instead, and install the script on that same box, okay? You will save yourself a big headache and there will be no chance that your logs get wiped out the next time your router reboots. Better still, keep an offline backup of your logs protected with lock and key, and send copies offsite over a VPN if you have a service like that available to you. You want to make sure you have evidence if someone messes with your network.
You should probably set your router to log all inbound and all outbound connections to make sure you're getting all the info (if, for example, you want to be able to see people who are talking to your internet facing services, instead of just people who are probing you).
You need to download this script, make a new folder /root/scripts, and copy the script into that new folder (obviously you will need root permission for everything, so obtain it and BE EXTREMELY CAREFUL, ONE MISTAKE AS ROOT AND YOU COULD LOSE EVERYTHING!).
After that, set the owner of the script to root and make it executable by going into the folder and issuing these commands:
chown root:root newiplookup.sh
chmod 700 newiplookup.sh
Now open the script in a text editor and change the values of the first four variables to some appropriate values (lines 31, 34, 37, and 42).
Now is the time to make sure you have followed my instructions properly. Test your setup by running the script in the background like this: /root/scripts/./newiplookup.sh&
and watch the log it creates by running: tail -f /root/.connection_log
Next, if it's your thing, you can use your mad scripting skills to tweak the log output so you get the info you need in the log.
Finally, you should install the script as a daemon that starts up automatically on system startup. The way this is accomplished differs between BSD and SysV derivatives, but usually what you do is copy a platform-specific init file into /etc/init.d
I am providing two example init.d scripts, so
if you're running Gentoo use this one: Gentoo init.d script
If you're running a distribution that uses Bourne Shell scripts for its init mechanism, such as
Debian or Ubuntu, use this one instead: Debian init.d script
Then make sure it's owned by root and executable by running:
chown root:root /etc/init.d/newiplookup
chmod 755 /etc/init.d/newiplookup
Now you only need to install the script into your desired runlevel.
On Gentoo, run the command: rc-update add newiplookup default
On Debian or Ubuntu, one way is to run the commands:
cd /etc/rc2.d
ln -s ../init.d/newiplookup S18newiplookup
Now you're really done! When you reboot, you will have a new logging facility that runs in the background all the time, giving you more meaningful logs that can really help you catch those bastards.
You can follow your new .connection_log in real-time with tail -f, or
you can switch to tty11 (crtl-alt-F11) to view the log there.
Once the database builds up and the logging slows down, it might be interesting to see how many unique IPs have tried to communicate with your network. A quick way to see the IP count in your "database" is to run the command: cat /root/.newiplookupdb | wc -l
Example Log Output
IPs and hostnames have been obfuscated for privacy in this example
2011-07-01 Fri 22:52:33 EDT IN 7x.2x3.x2.5x c-7x-2x3-x2-5x.hxdx.xa.cxmxaxt.nxt.
2011-07-01 Fri 22:53:59 EDT OUT x95.3x.x3.1x3 Host not found: 3(NXDOMAIN)
2011-07-01 Fri 22:54:45 EDT IN 9x.x02.1x1.x1x x1x.1x1.x02.9x.xfl.xex.rx.xom.
2011-07-01 Fri 22:55:04 EDT OUT x1.1x3.19x.1x3 xex1.taxloxstxrx.sx.
2011-07-01 Fri 22:59:28 EDT IN x9.2x6.14x.x4 xpe-x9-2x6-14x-x4.xvx.rxs.rx.cxm.
2011-07-01 Fri 23:00:12 EDT IN 2x9.x9.22x.x31 2x9-x9-22x-x31.xexstrxam.xtxa.xo.nx.
2011-07-01 Fri 23:00:15 EDT OUT x02.3x.24x.2x rexiaxt.nexgaxx.nex.xz.
2011-07-01 Fri 23:01:31 EDT IN 17x.x82.17x.x16 Host not found: 2(SERVFAIL)
2011-07-01 Fri 23:01:35 EDT OUT 1x2.1x9.x.2x4 nx1.xelxsqxebex.xex.
2011-07-01 Fri 23:01:38 EDT OUT x42.16x.x.x6 axexxs.qxebxcxex.cxx.
Enjoy!
Was this tutorial helpful? Toss me a few bucks if you're feeling generous.
Comments