The Roles of Intrusion Detection and Data Fusion in Cyber Security Situational Awareness

1. The Roles of Intrusion Detection and Data Fusion in Cyber Security Situational Awareness A Review of the Published Literature and Discussion of Future Research Plans Nicklaus A. Giacobe

2. Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion • Intrusion Detection (ID) Plays and Important Role in Developing Situational Awareness • Cyber Situational Awareness = • Network Security Situational Awareness • Activities Performed on Behalf of an Organization – “Network Security Office” • Activities Performed by Computer/Network Security Analysts • Difficult, Complex Work – Lots of Data from IDS, Antivirus Systems, Firewall Logs, Server Security Logs, etc. • Ever-Changing Landscape - New Threats, New Technologies, New Software, New Vulnerabilities

3. Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion This Introduction Part 1: What is the Current State of ID Technology? Part 2: What are We Trying to Accomplish? Part 3: Future Research Recommendations Conclusion/Discussion

4. Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations

5. Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations

6. History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Two Different Locations to Monitor Host-Based IDS (Denning) • Log Files (C2 compliance) on Unix Machines (Denning 1987) • IDES/NIDES – Baseline “normal” user behavior (Javitz et al. 1994) Network-Based IDS (Mukherjee/Heberlein) • NSM (LAN Monitor) – history of previous connections, known bad actors lists, signatures of attack types (Mukherjee et al. 1994) • NIDS (Multiple Network IDS and Host) (Snapp et al, 1991) (interesting JDL comparison)

7. History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Two Different Methods of Analysis • Pattern-Matching (Misuse) Detection (Spafford) • Match activity to patterns of known undesired • behavior (Kumar et al. 1994, 1995) • Tripwire – MD Hashing of files (Kim et al. 1994) • DDoS prevention /SYN Floods / Active DoS prevention (Schuba et al. 1997) • Anomaly Detection (Stolfo) • Looking for abnormalities in network traffic (Lee et al. 1999) • Qualitative evaluation of the data stream (statistical methods) (Portnoy, et al. 2001) – alert on infrequent types of data • Statistical Payload Evaluations – for Worm Detection (Wang et al. 2004, 2006a, 2006b) and mitigation (Locasto et al., 2006)

8. History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Testing and Evaluation of IDSs • DARPA IDS Data Sets from 1998-2000 • 1999 Data Set Contained • 2 Weeks of “training data” with labeled known intrusions • 7 Weeks of unlabeled data • Evaluate IDSs under design or in production • Over-fit problem • IDSs could be developed that find all of the problems in the “training data”, but could be very poor at alerting on novel intrusion methods

9. Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations

10. Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Correlate by Source, Destination or Attack Method Non-Trivial port-number vs. service name, IP address vs. hostname, etc. (Cuppens 2001) Need Adaptors – Different systems not designed for fusion (Debar et al. 2001) Promise of better understanding… see next slide

11. Understanding Through Correlation Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Adapted from (Debar et al. 2001)

12. Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion JDL Fusion Model (Hall and McMullen 2004)

13. Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion JDL Fusion Model (Hall and McMullen 2004) Source Pre-Processing Level 1 Object Refinement Level 2 Situation Refinement Level 3 Threat Refinement

14. Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualization of Underlying and Fused Data

15. Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Bayesian Inference • Complete list of all possible states of the system • Probabilities of current state • Need for accurate historical data (Holsopple et al. 2006) D-S Theory • No need for exact knowledge • Sort out independent evidence and combine it using the Dempster Rule • Very human-like logical combination • Can combine evidence of non-similar sources/data types

16. Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Data Mining Algorithms • Support Vector Machines (SVMs) (Liu et al. 2007 x3) • Neural Networks (Wang et al. 2007) • May be helpful in rapidly combining multiple sources of similar data • Thomas and Balakrishnan (2008) • Combined alert data from 3 different IDSs (PHAD, ALAD, Snort) using MLFF-NN • Tested vs. DARPA 1999 data set • Showed improved detection rates of the known data over each individual IDS (68% vs. 28%, 32%, 51%)

17. Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations

18. Visualizations Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Based on Network Topology Based on Geopolitical Topology Network Traffic Representations Alert and Track-Based Displays

19. Hierarchical Network Map from Mansmann and Vinnik (2006) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

20. Representation of Threats and Actors on a Geopolitical Map from (Pike et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

21. Representation of host to port to remote port to remote host of network traffic from (Fink et al. 2004) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

22. Panel Displaying Network Connections from a Single Host from (Fischer et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

23. Representing the Three Ws from (Foresti et al. 2007) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

24. Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

25. Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

26. Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

27. Definitions… Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion (Computer) Security is… Manunta (1999) • Security is interaction of Asset (A), Protector (P) and Threat (T) in a given Situation (Si) CIA Triad (Tipton et al. 2007) • Confidentiality • Integrity • Availability Bishop (2003) • Only authorized actions can be executed by authorized users

28. Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

29. Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Endsley (1995) State of Knowledge • Elements • Situation • Future Projection “Awareness Machine” unlikely • Focus instead on “awareness support technologies”

30. Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Endsley (1995)

31. Higher Levels of Fusion = Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Mapping of IDS Fusion tasks between JDL Model and Endsley SA Model. From Yang et al. (2009)

32. Higher Levels of Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion INFERD • Level 2 Fusion Engine – Based on a priori knowledge from system experts – pattern matching attack methods and known vulnerabilities of the system TANDI • Level 3 Fusion – Projection of future attacks based on knowledge of vulnerabilities of the system • (Yang et al. 2009)

33. Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

34. Cognitive Load Theory Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Sweller et al. (1998) • Working Memory (limited capacity) • Long Term Memory (unlimited capacity, based on schemas to represent complex, related information) • Split Attention • Conflicting, Repetitive • Modality Effect

35. Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

36. Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Biros and Eppich (2001) – CTA of IDS Analysts in the USAF - 5 capabilities required • ID non-local addresses • ID source addresses • Develop mental image of “normal” behavior • Create and maintain SA • Knowledge sharing Killcrece et al. (2003) – CTA of gov’t/military security specialists – 3 general categories • Reactive Work (majority of the work) • Proactive Work • Quality Management (training, etc)

37. Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion D’Amico et al. (2007) – CTA of Network Security Professionals in the Department of Defense

38. Part 3: Where Do We Go From Here? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Model Building • To understand the contributions of the algorithm builders CTA • To understand the needs of the analyst Visualization Recommendations • Based on the work above

39. Conclusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Current State of ID • History of ID • Alert Correlation and Data Fusion • Data fusion techniques • Visualization of underlying and fused data Theoretical Basis for Understanding SA in the Cyber Security Domain • Definition of Computer Security • Theory of Situational Awareness • Cognitive Load Theory • Cognitive Task Analysis Recommendations for Future Work • Model Building - To understand the contributions of the algorithm builders • CTA - To understand the needs of the analyst • Visualization Recommendations – Based on Needs and Cognitive Capabilities of Analysts

40. Discussion and Questions Just in case you needed a prompt to ask questions … here it is