Ourmon and Network Monitoring Performance

1. Ourmon and Network Monitoring Performance Jim Binkley jrb@cs.pdx.edu Bart Massey bart@cs.pdx.edu Portland State University Computer Science

2. ourmon introduction • ourmon is an open-source statistic gathering network monitoring system • with some similarities/differences to • traditional SNMP RMON II • name is a take off on this (ourmon is not RMON) • Linux ntop is somewhat similar, Cisco netflow, features different • we deployed it in the PSU DMZ a number of years ago (2001) • first emphasis on network stats • how many packets, how much TCP vs UDP, etc. • what the heck is going on out there? • recent emphasis on detection of network anomalies • post SQL slammer, Welchia worm, botnets

3. ourmon architectural overview • a simple 2-system distributed architecture • front-end probe computer – gather stats • back-end graphics/report engine • front-end depends on Ethernet switch port-mirroring • like Snort • does NOT use ASN.1/SNMP • summarizes/condenses data for back-end • intermediate data is ASCII • cp summary file via out of band technique • micro_httpd/wget, or scp, or rsync, or whatever

4. the Tao of ourmon • more information less data • summarization in probe OR back-end • 2.5 G packets (more later) means you need to summarize • no standards please other than • probe talks to back-end graphics/report makers in a shared language • probe and back-end always released together • we can change our intermediate data for the next release • new tuples of interest at any time

5. ourmon architectural breakdown pkts from NIC/kernel BPF buffer mon.lite report file probe box/FreeBSD graphics box/BSD or linux tcpworm.txt etc. outputs: 1. RRDTOOL strip charts 2. histogram graphs 3. various ASCII reports, hourly summaries or report period runtime: 1. N BPF expressions 2. + topn (hash table) of flows and other things (tuples or lists) 3. some hardwired C filters (scalars of interest) ourmon.conf config file filters: BPF expressions, lists, some hardwired C filters

6. features • N concurrent BPFs grouped into RRDTOOL graphs (about 80 expressions in DMZ now) • say 3-6 related BPF expressions per RRD graph • top-N tuples of various sorts: • traditional flows including flow counts, key is flow ID • top ports, key is TCP/UDP port (surprise) • TCP syn tuple: key is IP src • scanners/worms/P2P/IRC users, port report, tworm • ICMP/UDP error tuple: key is IP src • new IRC tuples (channels, per host stats) • IP and port scanning

7. BPF filter set: - TCP control pkts question: does your network syn curve match your fin curve?

8. tworm filter - DDOS/botnet-related attack ip src: flags work sa/s dst/s dst port/% 24.205.*.* (20) WOR 100 0 30/30 [30108,100]

9. UDP error filter - attack on single PSU media server err… attack is so large suppressed all other bars to < 1 pixel

10. test setup for ourmon/bpf measurement ixia 1600 gE packet generator ixia 1. min-sized pkts. 2 max-sized pkts ixia sends/recvs packets Packet Engines line-speed gigabit ethernet switch port-mirror of IXIA send port UNIX pc 1.7 ghz AMD 2000 UNIX/FreeBSD 64-bit PCI/syskonnect + ourmon/packet tap

11. ixia vs ourmon test summary • 1. 64 byte min packets bad; max large packets good • on ~2Ghz PIV, with no real work start losing min pkts at 80 mbits. • if you have real work (imagine snort with 1000’s of signatures) you may be 10 mbits • 2. big packets need bigger kernel buffers, say 8 megabytes, not 4k bytes • 3. actually dual box with NO software work is somewhat better

12. more ourmon information • ourmon.cat.pdx.edu/ourmon - PSU stats and download page • ourmon.cat.pdx.edu/ourmon/info.html - how it works (2 pages, 1 for current release and one for next release) • current release is 2.4 • next release will have more anomaly detection features including an automated trigger mechanism for pkt capture during “interesting” events (large attacks, UDP anomalies)