About

Wednesday, February 20, 2013

psad - log unsolicited internet traffic (too much of it)

Psad (Port Scan Attack Detector) is a useful daemon that offers a number of features and is very configurable. I'm always curious to know who has been probing my servers and psad along with some custom logging and other scripts seems to do this well.

Logging

 

Psad Log files
The psad daemon will poll iptables log files. It will then create a directory for each scanning IP in  /var/log/psad/xxx.xxx.xxx.xxx/ and log/lookup some information. The 11.11.11.11 IP  is your IP, you can have psad running on multiple IP addresses (great if they are in different ranges!) which will be logged too.
  • 11.11.11.11_email_alert - a summary of all information.
  • 11.11.11.11_packet_ctr - packets (INPUT_eth0_tcp:  51 [606] \n INPUT_eth0_udp:  69 [606]).
  • 11.11.11.11_start_time - first packet (1352391283).
  • XXX.XXX.XXX.XXX_whois - A whois lookup of the attacker.
  • danger_level - Different danger levels are configurable based on the number of packets.
  • email_ctr - Number of email alerts.
  • p0f_guess - Passive fingerprinting guess of OS (@Windows:XP:RFC1323:Windows XP/2000 (RFC1323 no tstamp)).
File system limitations reached
Thinking that I could build up some interesting data over time I configured psad not to remove any logs from old scans and left it running. After a couple of months I hit a limitation in the ext3 filesystem I was using, 31998 subdirectories. After checking ext4 did not offer that much of an advantage.

I settled on creating a ReiserFS partition to hold my psad logs. There is a noticeable performance improvement and it seems to be scaling well . I'm sure there are further tweaks to be done here.

Interesting Sources

 Completely unsolicited network traffic originates from some rather interesting places, for a wide variety of reasons.

A quick search of some terms that I could think of: 
$ find /var/log/psad/ -name "*_whois" -exec egrep -H -i "finance|bank|military|royal|defence|provider|industrial|health|government|\.gov|\.mil|\.aero" -m1 '{}' \;

Some of the results this search yielded:
  • whois: OrgName: Royal Military College of XXXXXX
  • whois: descr: Commercial Bank of XXXXXX
  • whois: descr: XXXXX Nation-wide Network of XXXXbank XX Corp.
  • whois: network: Org-Name; XXXXXHEALTHCARE
  • whois: descr: XXXXXX Internet Service Provider
The "College" may have been looking for sip ports, I logged x2 packets to udp 5060.

Responding

  • You can call an external script on a hit and pass the source IP to it. The psad.conf:
ENABLE_EXT_SCRIPT_EXEC      Y;
EXTERNAL_SCRIPT             /opt/scripts/psad_hai-back.sh SRCIP -v;
### (only once per IP, or every time a scan is detected for an ip).
EXEC_EXT_SCRIPT_PER_ALERT   N;
  • There is also the ability to automatically block (or do anything as you can specify the rules ran) an ip with iptables after a threshold. Real time black listing or redirecting:
ENABLE_AUTO_IDS N;
AUTO_IDS_DANGER_LEVEL 3:
### Based on Number of packets.
DANGER_LEVEL1               1;  
DANGER_LEVEL2               10;
DANGER_LEVEL3               100;
DANGER_LEVEL4               500;
DANGER_LEVEL5               1000; 
  • I have also been toying with inotify-tools and Sec (simple event correlation)  for custom logging and information gathering.
More on this coming.