1 / 45

What About Scanning?

This thesis defense explores the challenges of high bandwidth intrusion analysis and proposes a defense in depth solution using scanning and analyzing scan data. It discusses the limitations of current tools and methodologies, and suggests innovative approaches to improve intrusion detection in high bandwidth networks.

btotten
Download Presentation

What About Scanning?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What About Scanning? Analyzing Scan Data as part of a “Defense in Depth” Solution to the High Bandwidth Intrusion Detection Problem Douglas Cress

  2. The Way Ahead • Introduction and Motivation • Description of Scanning • Analyzing NIDS Alerts • Experiment Description • Conclusions and Future Work M.S. Thesis Defense 8/6/03

  3. High Bandwidth Intrusion Analysis Challenges • Class A networks have 16 million hosts, Class B networks have 65,535 hosts • Both class sizes require bandwidth in the Multiple T3 (45 Mb/s ~ 486 GB/day) to OC-3 (155 Mb/s ~ 1.67 TB/day) • Detecting Intrusions at line rate is basically impossible • Most NIDS only sample the data stream at such high bandwidths M.S. Thesis Defense 8/6/03

  4. High Bandwidth Intrusion Analysis Challenges • Small number of defenders vs. overwhelming force of attackers • Global Information Assurance Certification (GIAC) has certified only 643 people since 2000! • Constantly changing vulnerability landscape • 2,572 unique entries in the Common Vulnerability and Exposures (CVE) database • Ever increasing rise of non-mission essential software • P2P, Chat, Warez etc. M.S. Thesis Defense 8/6/03

  5. High Bandwidth Intrusion Analysis Challenges • Poor tools • Visualizations break down because of massive amount of data • Meta-data like CISCO NetFlow isn’t sufficient to prove an intrusion • Even Network Intrusion Detection Systems (NIDS), if poorly configured, can output more false alarms than true M.S. Thesis Defense 8/6/03

  6. Hacker Methodology • Information gathering – Scanning • Initial penetration – Buffer overflow • Privilege escalation – Password cracking • Various Activities – Data extraction • Attack Relay – Violate trust relationships M.S. Thesis Defense 8/6/03

  7. High Bandwidth Intrusion Analyst Solutions • Defense in Depth • Physical Devices • routers, firewalls, NIDS etc. • Organization security policies • Fair-use, virus scanning, etc. • Analysis methods • Real-Time, Trend, Area Of Responsibility (AOR), etc. M.S. Thesis Defense 8/6/03

  8. Defense in Depth HIDS NIDS Firewall Router M.S. Thesis Defense 8/6/03

  9. Thesis Synopsis • Reduce wasted analyst time by identifying most likely true-positive NIDS alerts based on related previous scanning • Using UMBC as a testing ground for theories • Novelty and Significance of work M.S. Thesis Defense 8/6/03

  10. Background TCP/IP • TCP, UDP, and ICMP are all susceptible to scanning • TCP has the three way handshake • SYN, SYN-ACK, ACK • UDP provides auto-response for available services • ICMP provides challenge and response functionality M.S. Thesis Defense 8/6/03

  11. Types of Scans • Scanning is not illegal • Moulton vs. VC3, 2000 • Half-open scan (aka SYN scan) • Null-host scan • OS scan • Packaged scan and attack tool M.S. Thesis Defense 8/6/03

  12. Scan Tools • NMAP (Network MAPer) • Most famous, most options • Nessus • One of many vulnerability scanners • Grim’s Ping • FTP – Warez emplacement tool M.S. Thesis Defense 8/6/03

  13. Generic NIDS Description • Network appliance designed to examine all passing traffic for embedded intrusions • Produces alarms / alerts for an analyst to review • Anomaly-based vs. Signature-based • Common Vendors include – ISS’s RealSecure, Cisco’s IDS, Enterasys’sDragon, and SNORT M.S. Thesis Defense 8/6/03

  14. Brief Description of SNORT • Open source – libpcap based • 3 parts • Packet decoder • Detection engine • Alert / logging system • SNORT pre-processors • stream4, conversation, and portscan2 M.S. Thesis Defense 8/6/03

  15. Parsing Logs • UMBC has over 15 million alerts a day • Use PERL to quickly parse logs to mine the most important information • Figure out who is involved in scanning (both source and destination IP) • Look for alerts either from or to IPs related to previously detected scanning M.S. Thesis Defense 8/6/03

  16. Predictive Analysis / Attack Forecasting • Data mining techniques are good for trend analysis • Type of scan should indicate skill level of attacker • SYN-scan perpetrated by worm or script-kiddie • Null-host scan wielded by skilled attacker M.S. Thesis Defense 8/6/03

  17. UMBC’s fitness as a Testing Ground • Class B address space (130.85.0.0/16) • Varied users and missions • Students, administrators, researchers • High bandwidth • Multiple T3’s • Small intrusion analysis group M.S. Thesis Defense 8/6/03

  18. Long-Term / Trend Analysis • Process of examining intrusion events over a long time period to determine both future events and missed past events • Difficult to perform • Massive amount of data to process and store • Urgency of the now often crowds out long-term view M.S. Thesis Defense 8/6/03

  19. November 2002 Raw Alerts M.S. Thesis Defense 8/6/03

  20. November 2002 Alert Types M.S. Thesis Defense 8/6/03

  21. November Top 5 per Day M.S. Thesis Defense 8/6/03

  22. Attack vs. Scan Alerts M.S. Thesis Defense 8/6/03

  23. Analysis Process • Execute scanTop10.pl against SNORT scan alerts • Execute checkAlerts2.pl to find SNORT attack alerts relating to the top ten scanning parties • Execute checkAlerts2_to_excel.pl to format the data for easy spreadsheet viewing M.S. Thesis Defense 8/6/03

  24. Nov 1 Top 10 Source Scanners M.S. Thesis Defense 8/6/03

  25. Nov 1 Top 10 Scan Victims M.S. Thesis Defense 8/6/03

  26. Nov 1 Scans vs. Month M.S. Thesis Defense 8/6/03

  27. Term Analysis for November • MY.NET.114.88 => ucommons-114-88.pooled.umbc.edu • MY.NET.170.176 => phaser.ucs.umbc.edu • MY.NET.150.213 => libpc11.lib.umbc.edu • MY.NET.150.220 => paladin.lib.umbc.edu M.S. Thesis Defense 8/6/03

  28. Term Analysis for November • Analysis focus for hosts involved in scanning and later attacking • Red Worm Alerts • x86 setuid exploit alarms • null scans M.S. Thesis Defense 8/6/03

  29. Four types of hosts • ucommons – Dynamically assigned • Could be anybody with a laptop • libpc11 – General use lab computer • Rotating user set • paladin – Personal use computer • Probably hacked • phaser – SA owned machine • Embarrassingly hacked? M.S. Thesis Defense 8/6/03

  30. Mar 1 Scans vs. Month M.S. Thesis Defense 8/6/03

  31. Term Analysis for March • MY.NET.97.29 => • ppp-29.dialup.umbc.edu • MY.NET.97.124 => • ppp-124.dialup.umbc.edu • MY.NET.97.148 => • ppp-148.dialup.umbc.edu • MY.NET.1.200 => Unresolved M.S. Thesis Defense 8/6/03

  32. Term Analysis for March • MY.NET.1.200 • Scanned with NMAP • Windows SMB attacks • Watch-listed host attempted access • Three Dial-up addresses all involved in IIS (Internet Information Server) attacks M.S. Thesis Defense 8/6/03

  33. Real-Time Illustration • November 11, 2002 • 1.2 million scans • Over 74,000 alerts • Boiled down to two hosts worth investigating • Discovered in less than five minutes M.S. Thesis Defense 8/6/03

  34. Nov 11th Scan & Attack Alerts M.S. Thesis Defense 8/6/03

  35. Nov 11th Scans correlated to Attacks M.S. Thesis Defense 8/6/03

  36. Real-Time Analysis Nov 11th • MY.NET.150.220 => paladin.lib.umbc.edu • Accessed over 1000 times by Dutch registered host • IIS overflow attempt • Possible Red Worm related activity M.S. Thesis Defense 8/6/03

  37. Real-Time Analysis Nov 11th • MY.NET.83.146 => aciv-83-146.pooled.umbc.edu • Probably wireless host • 250 Access attempts from different Dutch registered host • Further scanning against the UMBC host from a third Dutch host M.S. Thesis Defense 8/6/03

  38. Tools Created for Analysis • scanTop10.pl – examines SNORT scan logs and calculates the top 10 scanning offenders and victims • checkAlerts2.pl – compares the output of scanTop10.pl to a SNORT attack alert log • fit_checkAlerts2_to_excel.pl – formats the output from checkAlerts2.pl for absorption into a spreadsheet M.S. Thesis Defense 8/6/03

  39. Conclusions • My novel analysis method would help a small group of intrusion analysts tackle a large network’s NIDS logs • The analysis method is simple to perform and rapid in execution M.S. Thesis Defense 8/6/03

  40. Future Work • Integration of my analysis process into a SNORT Post-Processor would help reduce false-positives • SNORT already exports alerts in XML, is it possible to extend this feature to export alerts in RDFS or DAML+OIL to then be reasoned over in order to reduce false positives? M.S. Thesis Defense 8/6/03

  41. Future Work • Trend analysis is difficult because of the massive amount of data that must be stored. • Usually this data is stored in a compressed format which is then un-compressed during each search M.S. Thesis Defense 8/6/03

  42. Future Work • Perhaps storing a meta-rule version of the alerts which could then be reasoned over to provide a pointer into exactly the compressed file where the important events are located, would speed the information retrieval process M.S. Thesis Defense 8/6/03

  43. ??? Questions ???

  44. Selected Bibliography • S. Axelsson, “The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection.” In Proc. Of the 6th ACM Conference on Computer and Communications Security, 1999. • R. Bace, P. Mell, “Intrusion Detection Systems,” NIST Special Publication, Nov 2001, Available HTTP: http://csrc.nist.gov/publications/nistpubs/800-31/sp800-31.pdf • Honeynet Project, “Know Your Enemy: Statistics, Analyzing the past … predicting the future,” [Online Document], Jul 2001, [ cited 2003 Jun 25], Available HTTP: http://www.honeynet.org/papers/stats/ M.S. Thesis Defense 8/6/03

  45. Special Thanks • Dr. Nicholas for his help and mentoring • Andy Johnston for providing the SNORT logs and some background on UMBC • Paul Cress for his editing help M.S. Thesis Defense 8/6/03

More Related