Intrusion Detection Systems

1. Intrusion Detection Systems Meltem YILDIRIM 2004720361 05.05.2005 CmpE 526 – Operating System and Network Security

2. Agenda • Introduction to IDS • Classification of IDS • IDS Models • Available IDS Tools • Conclusion & Future Work

3. What is Intrusion? • Intrusion: Actions attempting to break into or misuse one’s system in violation of an established policy • Types of Intrusion: • Attempted break-ins • Masquarade attacks • Penetration of the security control system • Denial of Service • Malicious Use

4. What is an IDS? • IDS: system trying to detect and alert on attempted intrusions into a system or network • Reactive rather than proactive (usually does not prevent unauthorized users from entering the network, only identifies that an intrusion has occurred) • May provide diagnostic information, too • Objective: 100% accuracy • False positive: false alarm • False negative: letting an attack pass undetected

5. An IDS Protected Enterprise

6. Data Collection Issues Response Issues Elements of a Basic IDS Model • Audit Data (logs) • Keyboard inputs, command-based or application-based logs • Reference Data Store • Intrusion signatures (known attack patterns) • Profiles of normal behaviours • Algorithms searching for suspicious behaviour • Alarm

7. Classifying IDS’s • Offline v.s. Online • Host-Based v.s. Network-Based • Anomaly Detection v.s. Misuse Detection

8. Offline v.s. Online • Offline • audit data is processed periodically, not real-time • work on audit logs • data mining • Online • audit data is processed real-time continuously • may react and prevent an intrusion still going on

9. Host-Based v.s. Network-Based (1) • Host-Based / HIDS A SW installed on each node Disadvantage: Consume CPU time, storage, memory and other system resources

10. Host-Based v.s. Network Based (2) • Network-Based / NIDS • Monitors all packets on the network wire e.g. may watch for large number of TCP connection requests to many different ports • Either runs on a single machine (hub, router, etc.) or is divided into several sensors and one central analysis point • Usually utilize a network adapter • Typically host-independent but may be a SW package installed on a dedicated host • Monitors numerous hosts simultaneously but may suffer from performance problems as network speed increases

11. update profile statistically deviant? Attack State Audit Data System Profile generate new profiles dynamically Anomaly Detection v.s. Misuse Detection (1) • Anomaly Detection: • Assumption: “Attacks differ from normal behaviour” • Analyses the network or system and infers what is “normal” (Establishes a “normal activity profile”) • Interprets deviations from this “normal” behaviour as an intrusion • Profile generation: • one-time activity • current and previous profiles may be merged at intervals Activity measures such as CPU time used, number of network connections in a time period Adjustment of threshold levels is very important

12. Anomaly Detection v.s. Misuse Detection (2) • Anomaly Detection: • Advantages: • May catch novel attacks we have not seen before • Disadvantages: • Current implementations do not work very well (too many false positives/negatives) • Cannot categorize attacks very well • Difficult to train in highly dynamic environments • The system may be gradually trained by intruders

13. modify existing rules Rule match? Attack State Audit Data System Profile add new rules Anomaly Detection v.s. Misuse Detection (3) • Misuse Detection • Attacks are known in advance (signatures) • Matches signatures against the audit data stream • The attack signatures are usually specified as rules

14. Anomaly Detection v.s. Misuse Detection (4) • Misuse Detection • Advantages: • Easy to implement, deploy, update and understand • Low rate of false positives • fast • Disadvantages: • Cannot detect previously unknown attacks • Constantly needs to be updated with new rules • As good as the database of attack signatures

15. Anomaly Detection Misuse Detection IDS Models • Predective Pattern Generation • Fuzzy Classifiers • Neural Networks • Support Vector Machines • Expert Systems • Decision Trees • Keystroke Monitoring • State Transition Analysis • Pattern Matching • Autonomous Agents

16. p = 0.8 p = 0.15 p = 0.05 Predictive Pattern Recognition • Try to predict future events based on event history • e.g. Rule:E1 - E2→(E3 = 80%, E4 = 15%, E5 = 5%) E3 Intrusion: Left-hand side of the rule is matched but the right-hand side is statistically deviant from prediction E1 E2 E4 E5

17. MEDIUM HIGH MEDIUM HIGH MEDIUM LOW LOW 1 0 5 10 25 50 100 fuzzy space of 5 fuzzy sets Fuzzy Classifiers (1) data mining • No clear boundary between normal and abnormal events • Selection of features • Number of abnormal packets (invalid source or destination IP address) • Number of TCP connections • Number of failed TCP connections • Number of ICMP packets • Number of bytes sent / received per connection • …

18. Fuzzy Classifiers (2) • Detecting a Port Scan if count of UNUSUAL SDPs on port N is HIGH and count of DESTINATION HOSTS is HIGH and count of SERVICE Ports observed is MEDIUM-LOW then Service Scan of Port N is HIGH • Detecting a DoS Attack if count of UNUSUAL SDTs is HIGH and count of ICMPs is HIGH then DoS ALERT is HIGH SDP: source IP - destination IP - destination port SDT: source IP - destination IP - packet type

19. threshold inputs x1 w1 w2 x2 . . . output y Wn-1 n ? i = 1 xn-1 Σ xi · wi> threshold wn xn Neural Networks – IDS Prototypes (1) • Perceptron Model • simplest form of NN • single neuron with adjustable synapses (weights) and threshold • baseline for measuring the performance of other models

20. x1 x2 . . . xn input layer hidden layer output layer Neural Networks – IDS Prototypes (2) • Backpropagation Model • Multilayer feedforward network • input layer + at least one hidden layer + output layer • Correct detection rate ≈ 80% with 2% false alarms

21. Neural Networks – Data Preprocessing • 1st round: Selection of data elements protocol ID, source port, destination port, etc. • 2nd round: Creation of relational databases • 3rd round: Conversion of query results into an ASCII comma delimited normalized format supervised learning 0,2314,80,1573638018,-1580478590,1,1,401,3758,0 0,1611,6101,801886082,-926167166,1,1,0,2633,1

22. m i=1 m where Wi is the weight element Neural Networks – Detection Approaches (1) • Detection by Weight Hamming Distance • Let Vn = {0,1}n be the n-dimensional vector space over the binary field {0,1} where n = 0,1,…,∞ • Let A,B Є Vm ΣWi٠ (Ai⊗ Bi) • whd(A,B) = • Find WHD between • normal and current • behaviour. • If WHD > threshold • then ALARM

23. Effect of ICLN Update Rules Neural Networks – Detection Approaches (2) NEW! • Improved Competitive Learning Network • When a training example is presented to the network, the output neurons compete • Winning and losing neurons update their weight vector differently • Neurons become specialized to detect different types of attacks Learning rate desired – actual Δw = - η x (r - y) x Input

24. SVM / Support Vector Machines (1) F: n-dimensional feature space Training period: SVMs plot the training vectors in F and label each vector SVs make up a decision boundary in the feature space

25. num_SU_attempts 5 safe num_failed_logins 5 SVM / Support Vector Machines (2) e.g. n = 2 features num_failed_logins: number of failed login attempts num_SU_attempts: number of “su root” command attempts We feed the system with labeled vectors The system automatically draws the boundaries or hyperplanes by an algorithm

26. IF condition1 conditon2 ... THEN derived_fact1 derived_fact2 ... When the conditions are satisfied, the rule is activated. Antecedent Consequent Expert Systems (forward-chaining)

27. Sample Grammar for Expert Systems • BNF Grammar for Inference Rules • Variable Definition ‘VAR’ body_1 body_1 := var_name var_value var_value := list_of(value) | value • Detection Rules ‘RULE’ Id body_2 Id := value /* Id is the identifier of the rule */ Body_2 := list_of(condition) | condition ‘=>’ alert condition := feature operator term operator := contain | = | in | > | < term := value | list_of(value) | var_name • Action Rules ‘BEHAVIOUR’ body_3 body_3 := condition ‘=>’ action_argument condition := boolean expression action := update | log | exit | continue

28. Decision Trees • All nodes are represented by a tuple (C, R, F, L) • C = condition • (feature, operator, value) • R = set of candidate detection rules • F = feature set (already used to decompose tree) • L = set of detection rules matched at that node root = (null, All Rules, ∅, ∅) root C4.5: decision tree construction algorithm

29. Autonomous Agents • Several independent small processes operating and cooperating to maintain the system • Advantages • Efficiency • Fault tolerance • Extensibility • Scalability • Can be applied to Wireless Ad Hoc Networks • Disadvantage • Overhead of so many processes

30. Available IDS Tools • Commercial • RealSecure • Public-Domain • Shadow • Snort • Research Prototypes • Emerald

31. RealSecure • Real-time IDS • 3-part architecture • Network-based recognition engine • Monitors a network segment and look for packets that match attack signatures • Response: terminate connection, send alert, record session, reconfigure firewall • Host-based recognition engine • Analyses system logs • Response: terminate user processes, suspend user accounts • Administrator’s module • www.iss.net

32. Shadow • Composed of • Sensors • Reside at key monitoring points in network (outside firewall) • Extract packet headers save them to a monitoring file • Analysis station • Inside firewall • Reads the monitoring file periodically • joint venture of Naval Surface Weapons Center Dahlgren, Network Flight Recorder, the National Security Agency, and the SANS Institute • www.nswc.navy.mil/ISSEC/CID/

35. Snort • open-source public-domain ID tool • real-time traffic analysis and packet logging on IP networks • protocol analysis, content searching / matching • flexible rules language to describe traffic that it should collect or pass • large group of users who contribute new signatures • Installation guides written in Turkish! • www.snort.org

36. Emerald • Event Monitoring Enabling Responses to Anomalous Live Disturbances • hybrid method (anomaly + misuse detection) • users are grouped into independently administered domains (divide-and-conquer) • www.sdl.sri.com/emerald/

37. Conclusion & Future Work • Several IDS models either looking for attack signatures or abnormal behaviours (predictive pattern generation, NN, SVM, rule-based systems, decision trees, pattern matching, etc.) • Misuse detection methods are widely used in practice whereas Anomaly detection methods are more popular among researchers • Detecting a wider range of intrusions with fewer false negatives • Adaptation to modern networks with increased size, speed and mobility

38. Conclusion & Future Work • Further investigation in hybrid systems • Standardization for Interoperability IDMEF (Intrusion Detection Message Exchange Format) proposed by IETF • Defines the format of alerts and alert exhange protocols • Object-oriented representation, XML • http://www.ietf.org/internet-drafts/draft-ietf-idwg-idmef-xml-06.txt

39. References • Kemmerer, R.A., Vigna, G., Intrusion Detection: A Brief History and Overview, Security & Privacy, 2002. pp.27-30. • Sherif, J.S., Dearmond, T.G., Intrusion Detection: Systems and Models, Proceedings of the 11th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, 2002. • Abbes, T., Bouhoula, A., Rusinowitch, M., Protocol Analysis in Intrusion Detection Using Decision Tree, Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC’04), 2004. • McHugh, J., Christie, A., Allen, J., Defending Yourself: The Role of Intrusion Detection Systems, IEEE Software, 2000. • Liu, Y., Tian, D., Wang, A., ANNIDS: Intrusion Detection System Based on Artificial Neural Network, Proceedings of the 2nd International Conference on Machine Learning and Cybernetics, 2003. • Mukkamala, S., Janoski, G., Sung, A., Intrusion Detection Using Neural Networks and Support Vector Machines, 2002.

40. References • Mukkamal, S., Sung, A., Artificial Intelligent Techniques for Intrusion Detection, 2003. • Salameh, W.A., Detection of Intrusion Using Neural Networks, Studies in Informatics and Control, vol.13, no.2, June 2004. • Lei, J.Z., Ghorbani, A., Network Intrusion Detection Using an Improved Competitive Learning Neural Network, Proceedings of the 2nd Annual Conference on Communication Networks and Services Research, 2004. • Lindqvist, U., Porras, P., Detecting Computer and Network Misuse Through the Production-Based Expert System Toolset (P-BEST), Proceedings of the 1999 IEEE Symposium on Security and Privacy, 1999.