Introduction to Intrusion Detection

1. Introduction to Intrusion Detection Magnus Almgren Chalmers University of Technology

2. Outline of Presentation • My background • Justification for intrusion detection • Types of intrusion detection systems • Figures of merit • What to expect when deploying an IDS • State of the art • Discussion: Designing an IDS

3. My Background • Master of Science in Engineering Physics, Uppsala University • Thesis: Building an IDS for IBM Research, Zurich • A Lightweight Tool for Detecting Web Server Attacks, NDSS 2000 • Prototype for later security product • Master of Science in Computer Science with Distinction in Research, Stanford University (Fulbright Scholar) • Report: Web Server Security • Best Research Report, MS level at Stanford University • Application-Integrated Data Collection for Security Monitoring, RAID 2001 • Computer Scientist, SRI International • Research projects for mostly US military (DARPA, etc) for building better intrusion detection systems. Involved with intrusion detection research for about six years.

4. Justification for intrusion detection • Premise • We will never be able to have completely secure systems • Bugs in newly built software, and • legacy systems (db) used • An Intrusion Detection System monitors computer systems • Compare with a traditional surveillance camera • enforces your local security policy • IDS and Fraud Detection Systems similar • This talk mostly about IDS but will mention FDS

5. Detection What is an intrusion detection system? IDS key components • IDS are used for • Detect intrusions and intrusion attempts • Give alarms • Stop on-going attacks • Trace attackers • Investigate and assess the damage • Gather information for recovery actions Audit Source Models Response

6. Where does intrusion detection fit into the security landscape? • Principle: Defense in Depth (layered mechanisms) Prevent Detect React/Survive

7. Features of an IDS • An IDS is • Closely integrated into the system it is monitoring: • concerning current threats, and the • implementation of the monitored system. • Almost always designed and built after the monitored system is deployed. • Sometimes seemed as ad hoc. • A necessary complement to other types of protection mechanisms. • Even if you do not succeed to stop the intrusion, it is of value to know that an intrusion has indeed occurred, how it occurred and which damage that has been caused.

8. History of Intrusion Detection • First non-secret report, 1980 • Anderson, James P, Computer Security Threat Monitoring and Surveillance • Seminal Paper, 1987, SRI International • Denning, D E, An Intrusion-Detection Model • Field is still relatively new • Key questions remain to be answeredFor example: • How do you measure security?

9. Types of IDS • Traditionally divided in two different ways • Based on the placement (implying input/audit data) • Host-based versus network-based IDS • Based on the type of detection paradigm • Model normal versus anomalous behavior • Modern systems are a mixture • Hybrid system using both normal and anomalous models, and • application-based, etc.

10. Traditional Host IDS • IDS deployed on the host it monitors • Reliable source of local events (processes, uid, paths), but • goes down with the monitored host. • Data sources • Audit data – useful, system call information dominates, but limited insight into application data • Application/system log files – limited content, disk space management • Usually, data produced after the fact • Blind to most network-level attacks • Administrative overhead • With many resources to protect, also many IDS need to be deployed and administered.

11. Advantages: Passive, non-invasive Hidden Can monitor multiple hosts from one location Problem areas: Encrypted traffic Evasion tricks Network speed Session/transaction reconstruction and statefulness Timely preemption difficult Traditional Network IDS

12. Sound alarm, if payload contains/..%c0%af../ Normal behavior Number Detection Models • Misuse detection (signature-based) • Define what is wrong and give alarms for such behavior.(default permit) • Anomaly Detection • Define what is correct and give alarms for everything else.(default deny)

13. Attack Space Detection Coverage • The detection coverage of the same type of IDS is similar. • NIDS examples • Snort, ISS, Dragon, SRI’s eXpert-Net NIDS HIDS • Different IDS types complement each other somewhat.

14. Ref Model Generation Process self-organizing maps self- learning anomaly NN hybrid NN ripper eStat eBayes spec- based bottleneck verification eXpert-BSM programmed snort webIDS Ref Model Type misuse anomaly IDS Survey:Some Research Systems

15. no alarm alarm Miss OK intrusion OK False Alarm no intrusion Figures of Merit for IDS • No alarms should be given in the absence of intrusions. • Intrusion (attempts) must be detected. • Probability of detectionhit rate • Rate of false positivesfalse alarm rate • Rate of false negativesmiss rate

16. Theoretical Baseline:Base Rate Fallacy • Bayesian Detection Rate • Intrusions are uncommon, which means that even if the false alarm rate is low you will still have many false alarms.

17. Using lower protocol levels [Ptacek and Newsham] Crafting ambiguous HTTP request GET /cgi-bin/phf GET /%00cgi-bin/phf GET / HTTP 1.1Host: victimContent-Length: 3123GET /cgi-bin/phf The evasion techniques work because Web servers and NIDS decode them differently Evasion Techniques: Web example Tab

18. Other Problems: Alert Inundation • SYN flood attack (Denial-of-service) • Early systems: 1 alert per SYN packet • Today better … • Port Scans • Happens all the time, but we do not care • Ineffective Attacks • Code Red against my Apache server Low Abstraction Level of Alerts:Several SYN packets received  We are under a DoS attack

19. Popular research area: Alert Handling • Alert Aggregation • Simple alerts aggregated within a sensor • Alert Correlation • More complex correlations, often between different types of sensors. • Interoperability with IDMEF • Topology Vetting • By knowing the topology of the network and which services are installed, we can avoid harmless alerts. • Alert Prioritizing • Prioritize alerts based on the interest of the operator.

20. Alert Handling Models Event Stream Maturity: Commercial Research Plan Recognition Filtering & Aggregation Correlation Sensors

21. Deploying an IDS • IDS != FW • Not something you just drop into your network and then leave there • Extensive tuning needed • Some regular analysis of all alarms should be expected • Compare with a surveillance camera.Not much use without a guard watching it! • One difference IDS versus FDS • Usually more people handling FDS alerts • Economic cost easier to calculate, and that justifies people’s salary

22. State of the Art • Gartner: IDS is dead. Long live IPS. • Catches well-known attacks and • may even catch simple variants. • Do not detect • new attacks, • larger variants, and • the Insider (basically ignored). • Especially true, if masquerades: uses a little bit more than he should of all resources. • False alarms are a major problem. • Mostly misuse systems, and very few anomaly detection systems among the commercial systems.

23. 1st Prevent intrusions 2nd Detect, blockintrusions 3rd Operatethrough attacks 4th Predict, diagnose,heal, and improve DARPA’s Four Generations of Security and Survivability Technology • Trusted Computing Base • Encryption • Authentication and access control • Multi-level security • Boundary Controllers • Intrusion Detection Systems • Public Key Infrastructure • Virtual Private Networks • Real-time execution monitoring • Intrusion tolerance • Error Detection • Error Compensation • Graceful degradation • Self-regenerative systems • Reliability is continually improved • Self-optimization, self-diagnosis, self-healing • Critical services never lost

24. Future Directions of Research • AI & IDS Revival in the US • More self-aware systems that can self-heal • Anomaly-based systems • My paper: Active Learning and IDS • Formally define systems and their detection capabilities • Handling gigabit speeds • Alert handling • Less of black box solutions

25. Relevant Papers for Intrusion Detection • Lightweight Tool for Detecting Web Server Attacks • Application-Integrated Data Collection for Security Monitoring • Malicious Code Outbreak Discovery: Issues and Approaches • An Architecture for an Adaptive Intrusion Tolerant Server • Dependable Intrusion Tolerance: Technology Demo • Implications of IDS Classification on Attack Detection • Using Active Learning for Intrusion Detection (most found on my home page) time http://www.ce.chalmers.se/staff/almgren

26. Further Reading • Amoroso: Intrusion DetectionIntrusion.net Books, ISBN 0-9666700-7-8 • Escamilla: Intrusion DetectionWiley, ISBN 0-471-29000-9(mainly about networking and firewalls) • Bace: Intrusion Detection, MTP, ISBN 1-57870-185-6 • Lindqvist: On the Fundamentals of Analysis and Detection of Computer Misuse (phd thesis), Department of Computer Engineering, Chalmers, ISBN 91-7197-832-1 • Northcutt et al • Network Intrusion DetectionISBN 0735712654 • Intrusion Signatures and AnalysisISBN 0735710635 • Inside Network Perimeter Security: The Definitive Guide to Firewalls, Virtual Private Networks (VPNs), Routers, and Intrusion Detection SystemsISBN 0735712328