1 / 32

Defending Distributed Systems Against Malicious Intrusions and Network Anomalies

Defending Distributed Systems Against Malicious Intrusions and Network Anomalies. Kai Hwang Internet and Grid Computing Laboratory University of Southern California

hamlin
Download Presentation

Defending Distributed Systems Against Malicious Intrusions and Network Anomalies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Defending Distributed Systems Against Malicious Intrusions and Network Anomalies Kai Hwang Internet and Grid Computing Laboratory University of Southern California Keynote Presentationat the IEEE International Workshop on Security in Systems and Networks (SSN-2005),held in conjunction withthe IEEE International Parallel and Distributed Processing Symposium (IPDPS-2005),Denver, Colorado, April 8, 2005 This presentation is based on research findings by USC GridSec team. Project Web site: http://GridSec.usc.edu, supported by NSF ITR Grant No. 0325409, and contributed by Min Cai, Shanshan Song, Ricky Kwok, Ying Chen, and Hua Liu

  2. Presentation Outline: • Security/privacy demands in networked or distributed computer systems • GridSec NetShield architecture for defending distributed resource sites in Grids, clusters, etc. • Internet datamining for collaborative anomaly and intrusion detection system (CAIDS) with traffic episode rule training and analysis • Fast containment of internet worm outbreaks and tracking of related DDoS attacks with distributed-hashing overlays

  3. Security and Privacy Demands inNetwork and Distributed Systems • Trusted resource allocation, sharing, and scheduling • Secure communications among resource sites, clusters, and protected download among peer machines • Intrusion and anomaly detection, attack repelling, trace back, pushback of attacks, etc • Fortification of hardware/software (firewalls, packet filters, VPN gateways, traffic monitors, security overlays, etc. ) • Self-defense toolkits/middleware for distributed defense, risk assessment, worm containment, response automation • Anonymity, confidentiality, data integrity, fine- grain access control, resolving conflicts in security policies, etc

  4. Site S1 3 Host VPN Gateway Internet 3 Host 3 Host 2 3 3 Host Host 2 3 3 Host Host 1 3 VPN Gateway 3 3 VPN Gateway Host Host Site S3 Site S2 Steps for automated self-defense at resource site : Step 1: Intrusion detected by host-based firewall /IDS Step 2: All VPN gateways are alerted with the intrusions Step 3: Gateways broadcast response commands to all hosts GridSec: A Grid Security ITR Project at USC

  5. The NetShield Architecture with Distributed Security Enforcement over a DHT Overlay

  6. Building Encrypted Tunnels between Grid Resource Sites Through the DHT Overlay • The number of encrypted tunnels should grow with O(N) instead of O(N x N), where N is the number of Grid sites • Using shortest path, security policy is enforced with minimal VPN tunnels to satisfy special Grid requirements, automatically • How to integrate security policies from various private networks through the public network ? • How to resolve security policy conflicts among hosts, firewalls, switches, routers, and servers, etc. in a Grid environment ?

  7. Site S3 Site S2 Site S1 Physical backbone DHT Overlay Ring Trust Vector Trust vector propagation User application and SeGO server negotiation Site S4 V V V V V VPN Gateway SeGO Server Hosts Cooperating gateways working together to establish VPN tunnels for trust integration Trust Integrationover a DHT Overlay

  8. Datamining for Anomaly Intrusion Detection (IDS) Network Router ISP Firewall Victim’s Internal Network The NetShield System The Internet Intrusion ResponseSystem (IRS) Risk Assessment System (RAS) USC NetShield Intrusion Defense System for Protecting Local Network of Grid Computing Resources

  9. Alert Operations performed in local Grid sites and correlated globally

  10. Basic Concept of Internet Episodes • Event Type: A, B, C, D, E, F, etc. • Event Sequence: e.g., <(E,31),(D,32),(F,33)> • Window: Event sequence with a particular width • Episode: partially ordered set of events, e.g. whenever A occurs, B will occur soon • Frequency of episode: fraction of windows in which episode occurs • Frequent episode: set of episodes having a frequency over a particular frequency threshold • Frequent episode rules are generated to describe the connection events

  11. Frequent Episode Rules (FER) for CharacterizingNetwork Traffic Connections E→D, F ( c, s ) The episode of 3 connection events (E, D, F) = (http, smtp, telnet). On the LHS , we have the earlier event E (http). On the RHS, we have two consequence events D (smtp) and F(telnet); where s is the support probablity and c is the confidence level specified below:(service = http, flag = SF) → (service = smtp, srcbyte = 5000), (service = telnet, flag = SF) (0.8, 0.9) Support probability s = 0.9 and Confidence level c = 0.8 that the episode will take place in a typical traffic stream

  12. Training data from audit normal traffic records Single-connection attacks detected at packet level ADS Episode Rule Database IDS Audit records from traffic data Unknown or burst attacks Anomalies detected over multiple connections Signature MatchingEngine Episode Mining Engine ADS Known attack signatures from ISD provider New signaturesfrom anomalies detected Attack Signature Database Signature Generator A Cooperative Anomaly and Intrusion Detection System (CAIDS), built with a Network Intrusion Detection System (NIDS) and an Anomaly Detection System (ADS) operating interactively through automated signature generation

  13. Internet Datamining for Episode Rule Generation

  14. Attack Spectrum from MIT Lincoln Labin 10 Days of Experimentation

  15. 1. Label relevant connections toassociate with an FER. Online traffic episode rules from the datamining engine Episode rules matching the normal FER database ? Yes Episode Frequency exceeding the rule threshold ? No (Stealthy attacks) No • Check error flags or other useful temporal statistics • Extract common features suchas IP addresses, protocol, etc.to form the signature Yes (Massive attacks) • Calculate additional information such as connection count, average and percentage of connections, etc. • Select one of the predefined classifiers • Use the selected classifier to classify the attack class and find the relevant connections • Extract common features in all identified connections, such as the IP addresses, protocol, etc. to form the signature Adding new signatures to the Snort database Ignore the normal episode rules from legitimate users (No anomaly detected) Automated Signature Generation from Frequent Episode Analysis

  16. Successful Detection Rates of Snort , Anomaly Detection System (ADS), and the Collaborative Anomaly and Intrusion Detection System (CAIDS)

  17. False Alarms out of 201 Attacks in CAIDS Triggered by Different Attack Types under Various Scanning Window Sizes Using larger windows result in more false alarms. Shorter windows in 300 sec or less are better in the sense that shorter episodes will be mined to produce shorter rules, leading to faster rule matching in the anomaly detection process

  18. Detection Rates of Snort, ADS, and CAIDSunder Various Attack Classes On the average, the CAIDS (white bars) outperforms the Snort and ADS by 51% and 40%, respectively

  19. ROC Curves for 4 Attack Classes on The Simulated CAIDS

  20. ROC Performance of Three Intrusion Detection Systems

  21. Internet Worm and Flood Control: • A DHT-based WormShield overlay network is under development at USC. • Fast worm signature generation and fast dissemination through both local and global address dispersion • Automated tracking of DDoS attack-transit routers to cut off malicious packet flows for dynamic DDoS flood control

  22. The WormShield Built with a DHT-based Overlay with Six Worm Monitors

  23. The WormShield Signature Generation Process

  24. Signature Detection in Worm Spreading and the Growth of Infected hosts for Simulated CodeRed Worms on a Internet Configuration of 105,246 Edge networks in 11,342 Autonomous Systems Containing 338,652 Vulnerable Hosts

  25. Effects of Local Prevalence ThresholdWorm spreading and the growth of infected hosts

  26. Effects of Global Address Prevalenceon Worm Spreading and the Growth of Infected Hosts

  27. Reduction of Infected Hosts by Independent vs. Collaborative Monitoring over the Edge Networks

  28. Packet/Flow Counting for Tracking Attack-Transit Routers (ATRs)

  29. False Positive Rate of Identified ATRs

  30. Other Hot Security Research Areas: • Efficient and enforceable trust models are very much in demand for networked and distributed systems: PKI services, VPN tunneling, trust negotiation, security overlays, reputation system etc. • Large-scale security benchmark experiments in open Internet environments are infeasible. The NSF/HSD DETER testbed should be fully used in performing such experiments to establish sustainable cybertrust over all edge networks. • Internet datamining for security control and for the guarantee of Quality-of-Service in real-life network applications – Interoperability between wired and wireless networks is a wide-open area for further research.

  31. Final Remarks • The NetShield built with DHT-based security overlay networks support distributed intrusion and anomaly detection, alert correlation, collaborative worm containment, and flooding attack suppression. • The CAIDS can cope with both known and unknown network attacks, secure many cluster/Grid/P2P operations in using common Internet services: telnet, http, ftp, Email, SMTP, authentication, etc. • Automated virus or worm signature generation plays a vital role to monitory network epidemic outbreaks and to give early warning of large-scale system intrusions, network anomalies, and DDoS flood attacks. Extensive benchmark experiments on the DETER test bed will prove the effectiveness.

  32. Recent Related Papers: • M. Cai, K. Hwang, Y. K. Kwok, Y. Chen, and S. S. Song, “Fast Containment of Internet Worms and Tracking of DDoS Attacks with Distributed-Hashing Overlays”, IEEE Security and Privacy, accepted to appear Nov/Dec. 2005. • K. Hwang, Y. Kwok, S. Song, M. Cai, R. Zhou, Yu. Chen, Ying. Chen, and X. Lou, “GridSec: Trusted Grid Computing with Security Binding and Self-Defense against Network Worms and DDoS Attacks”, International Workshop on Grid Computing Security and Resource Management (GSRM’05), in conjunction with ICCS 2005, Atlanta, May 22-25, 2005. • M. Qin and K. Hwang, “Frequent Episode Rules for Internet Traffic Analysis and Anomaly Detection”, IEEE Network Computing and Application Symp. (NCA-2004), Cambridge, MA. August 31, 2004 • K. Hwang, Y. Chen and H. Liu, “ Defending Distributed Computing Systems from Malicious Intrusions and Network Anomalies”, IEEE Workshop on Security in Systems and Networks (SSN’05), in conjunction with IEEE IPDPS 2005, Denver, April 8, 2005.

More Related