Network-based Intrusion Detection, Prevention and Forensics System

1. Yan Chen Department of Electrical Engineering and Computer Science Northwestern University Lab for Internet & Security Technology (LIST) http://list.cs.northwestern.edu Network-based Intrusion Detection, Prevention and Forensics System

2. The Spread of Sapphire/Slammer Worms

3. Current Intrusion Detection Systems (IDS) • Mostly host-based and not scalable to high-speed networks • Slammer worm infected 75,000 machines in <10 mins • Host-based schemes inefficient and user dependent • Have to install IDS on all user machines ! • Mostly simple signature-based • Cannot recognize unknown anomalies/intrusions • New viruses/worms, polymorphism

4. Current Intrusion Detection Systems (II) • Cannot provide quality info for forensics or situational-aware analysis • Hard to differentiate malicious events with unintentional anomalies • Anomalies can be caused by network element faults, e.g., router misconfiguration, link failures, etc., or application (such as P2P) misconfiguration • Cannot tell the situational-aware info: attack scope/target/strategy, attacker (botnet) size, etc.

5. Network-based Intrusion Detection, Prevention, and Forensics System • Online traffic recording [SIGCOMM IMC 2004, INFOCOM 2006, ToN 2007, INFOCOM 2008] • Reversible sketch for data streaming computation • Record millions of flows (GB traffic) in a few hundred KB • Small # of memory access per packet • Scalable to large key space size (232 or 264) • Online sketch-based flow-level anomaly detection [IEEE ICDCS 2006] [IEEE CG&A, Security Visualization 2006] • Adaptively learn the traffic pattern changes • As a first step, detect TCP SYN flooding, horizontal and vertical scans even when mixed • Online stealthy spreader (botnet scan) detection [IWQoS 2007]

6. Network-based Intrusion Detection, Prevention, and Forensics System (II) • Polymorphic worm signature generation & detection [IEEE Symposium on Security and Privacy 2006, IEEE ICNP 2007] • Accurate network diagnostics [ACM SIGCOMM 2006] [IEEE INFOCOM 2007 (2)] • Scalable distributed intrusion alert fusion w/ DHT [SIGCOMM Workshop on Large Scale Attack Defense 2006] • Large-scale botnet and P2P misconfiguration event forensics [work in progress]

7. RAND system RAND system Internet Internet scan port LAN Internet LAN RAND system LAN Switch Switch Splitter Switch Splitter Router Router Switch Switch Router scan port LAN LAN Switch LAN (a) HPNAIDM system (b) (c) System Deployment • Attached to a router/switch as a black box • Edge network detection particularly powerful Monitor each port separately Monitor aggregated traffic from all ports Original configuration

8. P2P Doctor: Measurement and Diagnosis of Misconfigured Peer-to-Peer Traffic Anup Goyal, Zhichun Li, Yan Chen and Aleksandar Kuzmanovic Lab for Internet and Security Technology (LIST) Northwestern Univ.

9. What is P2P Misconfiguration • P2P file sharing accounted for > 60% of traffic in USA and > 80% in Asia • Thousands of peers send P2P file downloading requests to a “random” target on the Internet • possibly triggered by bugs or by malicious reasons • generates large amount of unwanted traffic • It contributes on an average of about 30% of the “Internet background radiation”

10. Motivations • On Dec. 6th, 2006, 5,047sources generated >31,000 packets/sec and 11MB/s of traffic to a single unused IP in Northwestern University • P2P software DC++ has already been exploited by attackers for DoS • direct gigabit “junk” data per second to a victim host from more than 150,000 peers • Currently, little is known about the characteristics or root causes of P2P misconfiguration events 11MB/s

11. Outline • Motivation • Passive measurement results • P2P Doctor system design • Root cause diagnosis and analysis • Conclusion

12. Peer Classification Poisoned Peers (Intentional) Unintentionally Misconfigured peers All the peers Normal Peers Bogus Peers Anti-P2P Peers Not in the P2P Network In the P2P Network

13. Passive Measurement • Honeynet/honeyfarm datasets • Events: # of unique sources > 100 in 6 hours • After filtering scan traffic • Event characteristics: • Mostly target a single IP • Duration: A few hours to up to a month

14. 30%! Popularity • Growth Trend: • IP space • Observed in three sensors in five different /8 IP prefixes The average total connections of P2P misconfiguration events per month.

15. Further Diagnosis • Problems with passive measurement on archived data • Events have gone • Hard to backtrack the propagation • Root cause? • Need a real-time backtracking and diagnosis system!

16. Outline • Motivation • Passive measurement results • P2P Doctor system design • Root cause diagnosis and analysis • Conclusion

17. 10100101011101 infohash; ‘abc.avi’ Design of P2P Doctor System P2P-enabled Honeynet Backtracking system Root cause inference P2P payload signature based responder Event identification Protocol parsing for metadata

18. Design of P2P Doctor System P2P-enabled Honeynet Backtracking system Root cause inference Index Server (tracker) Crawling BT: top 100, eMule: 185 DHT Crawling Peer Exchange Protocol Crawling

19. Design of P2P Doctor System P2P-enabled Honeynet Backtracking system Root cause inference • What is the root cause? • Which peers spread misconfigurtion? • How is misconfiguration disseminated? • What is the percentage of bogus peers in the misconfigured P2P networks?

20. Deployment and Data Collection • Deployed the P2P doctor system on NU honeynet (10 /24 networks in three /8) • Real-time events • Previous passive measurement data referred as historical events

21. Outline • Motivation • Passive measurement results • P2P Doctor system design • Root cause diagnosis and analysis • Conclusion

22. Root Cause Analysis • Methodology • Track how honeynet IPs propagated in P2P systems • Use unroutable IP space as a big honeynet (66.8% of IPv4 Space) • Hypothesis formulation and testing • Classification of measured peers • Misconfigured peers: Passively observed from honeynet • Backtracked peers: actively observed through backtracking • Reverse honeynet peers: the IP obtained by reversing the target IP from the honeynets • Results • Data plane traffic radiation • Detailed results focus on eMule and BitTorrent

23. Peer Exchange DHT Index Server Data Plane Traffic Radiation 1.2.3.4 Resource mapping Who has Beowulf.avi? 1.2.3.4

24. eMule – Root Cause • Byte ordering is the problem! 4.3.2.1 1.2.3.4 1.2.3.4 4.3.2.1 4.3.2.1 4.3.2.1 4.3.2.1

25. eMule – Root Cause • Byte ordering is the problem! • Hypothesis from the historical data • In 80% of events, the reverse target IPs are alive • Verified with real-time events • 61% of the reverse honeynet peers indeed running eMule with the port number reported • For the backtracked peers which is in the unroutable IP space, 69.6% of them having reverse IPs run eMule

26. eMule – Peers & Dissemination • Which peers spread misconfiguration? • 99.24% of misconfigured peers are normal peers • How is the misconfiguration disseminated? • Index Server? No • Peer exchange? Yes • Percentage of bogus peers in eMule network? • [12.7%, 25.0%] w/ a total of 37,079 backtracked peers

27. BitTorrent – Responsible Peers • Both anti-P2P and normal peers are responsible • Events classified to two types with diagonally different sets of characteristics • For anti-P2Ppeers events • All the sources are from the IP range owned by anti-p2p companies like Media Defender, Media Sentry, Net Sentry etc. • Seen 6 out of 7 major anti-P2P companies sources in our honeynet.

28. Refuted Byte Ordering Hypothesis For 20 real-time events, no reverse honeynet peers runs BitTorrent For normal peer events, culprit is Peer Exchange (PEX) protocol implemented by uTorrent-compatible clients For anti-P2P peer events Possibly related to Azureus system Still an open question (No real-time events) BitTorrent – Root Cause

29. BitTorrent – Dissemination • How is misconfiguration disseminated? • Index server? - No • Peer exchange? - Yes • Percentage of bogus peers in BitTorrent network? • Out of a total of 9,000backtracked peers, only 13IPs are unroutable and 3,150IPs gave connection timeout • 0.14%< bogus Peers < 35%

30. Conclusions • The first study to measure and diagnose large-scale P2P misconfiguration events • Found 30% Internet background radiation is caused by P2P misconfiguration • Popular in various P2P systems, exponential growth trend, and scattered in the IPv4 space • For eMule, we found it is caused by network byte order problem • For BitTorrent, classified to anti-P2P peer events and normal peer events with diagonally different sets of characteristics • Found the uTorrent PEX causes the problem in normal peer events

31. ? ? ?

32. Backup Slides

33. Motivation • Given unprecedented amount of traffic, even a slight mis-configuration of the P2P system can result in a DDoS kind of situation • Prevalence in time, space, and across a number of distinct P2P systems with a temporal increasing trend is alarming. • P2P miscongurations can cause innocent people to get involved in the above “war” between P2P and anti-P2P systems. • Presently, nothing is known about the causes or overall effects of P2P mis-configurations • Our goal is to determine the root cause(s) of each type of mis-configuration

34. Related Work • Misconguration is widely spread across different networked and distributed systems like BGP [Labovitz et al. ]and firewalls [Cuppens et al. ]. • Measurement studies of normal P2P traffic [ACM SOSP (2003), MCN (2002)], while we measure the abnormal P2P traffic observed in honeynets. • In [INFOCOM (2005)], Content pollution including intentional and unintentional pollution is widespread for popular titles. • P2P systems like Fasttrack and Overnet are vulnerable to the index poisoning attack [INFOCOM (2006)] • All of the above studies focus on the content pollution or index poisoning while our focus is the index misconfiguration. • First large-scale measurement study on the root causes for both intentional/unintentional indexmisconfiguration.

35. What is P2P Misconfiguration • More than 50% of the traffic in the Internet today is P2P traffic • By Symantec Corporation’s recent report • P2P file sharing accounted for > 60% of traffic in USA and > 80% in Asia