Hybrid Intelligent Systems for Network Security

1. Hybrid Intelligent Systems for Network Security Lane Thames Georgia Institute of Technology Savannah, GA lane.thames@gtsav.gatech.edu

2. Presentation Overview • Discuss the goals of this project • Overview of Self Organizing Maps • Overview of Bayesian Learning Networks • Describe the details of the Hybrid System • Review the Experimental Results • Discuss Conclusions and Future Work • Q&A

3. Internet Growth • Internet Growth is Steadily Increasing • Many different types of applications are now using the Internet as a communication channel

4. Data Source: www.idc.com

5. The life of a network security professional

6. Data Source: http://www.cert.org/stats/cert_stats.html

7. Current Issues with Security • Short time between disclosure of vulnerability and attack • Huge Rule Base • Huge Signature Databases • Lag time between attack detection and signature creation • Lag time between vulnerability discovery and patch deployment

8. Project Goals • Develop an Intelligent System that works reliably with data that can be collected purely within a Computer Network • Why? If security mechanisms are difficult to use, people will not use them. • Using data from the network takes some of the burden off the end user

9. Hybrid Intelligent Systems • A system was developed that made use of two types of Intelligence Algorithms: • Self-Organizing Maps • Bayesian Learning Networks

10. Training and Testing Data Set • KDD-CUP 99 Data Set • The Data set used for the Third International Knowledge Discovery and Data Mining Tools Competition

11. Training and Testing Data Set • 41 Total Features Categorized as: • Basic TCP/IP features • Content Features • Time Based Traffic Features • Host Based Traffic Features

12. Self Organizing Maps—SOM • Pioneered by Dr. Teuvo Kohonen • An algorithm that transforms high dimensional input data domains to elements of a low dimensional array of nodes

13. Self-Organizing Maps • Input Data Vectors • Parametric Vector associated with each element, i, of the grid

14. Self-Organizing Map • A decoder function is defined on the basis of distance between the input vector and the parametric vector. • The decoder function is used to map the image of the input vector onto the SOM grid. The decoder function is usually chosen to be either the Manhattan or Euclidean distance metric.

15. Self-Organizing Maps • A Best Matching Unit, denoted as the index c, is chosen as the node on the SOM grid that is closest to the input vector

16. Self-Organizing Maps • The dynamics of the SOM algorithm demand that the Mi be shifted towards the order of X such that a set of values {Mi} are obtained as the limit of convergence of the following:

17. Bayesian Learning Networks—BLN • A BLN is a probabilistic model, and the network is built on the basis of a Directed Acyclic Graph (DAG) • The directed edges of the graph represent relationships among the variables

18. Bayesian Learning Networks • The Fundamental Equation: Bayes Theorem

19. Bayesian Learning Networks • In Bayesian learning, we calculate the probability of an hypothesis and make predictions on that basis

20. Bayesian Learning Networks • With BLN, we have conditional probabilities for each node given its parents • The graph shows causal connections between the variables • Prediction and abduction x4

21. Naïve Bayesian Learning Network • The Naïve BLN is a special case of the general BLN • It contains one root node which is called the class variable, C • The leaf nodes are the attribute variables (X1 … Xi) • It is Naïve because it assumes the attributes are conditionally independent given the class x1

22. The Naïve BLN Classifier • Once the network is trained, it can be used to classify new examples where the attributes are given and the class variable is unobserved—abduction • The Goal: Find the most probable class value given a set of attribute instantiations (X1 … Xi)

23. Hybrid System Details SOM Training Training Data Subset

24. Hybrid System Details Trained SOM Data Modified Data BN Development Module

25. Hybrid System Details BN Development Module Structure File Training Data Bayesian Training

26. Hybrid System Details Bayesian/SOM Classifier Test Data Classification File

27. Experimental Results • 4 types of analyses were made with the dataset • BLN analysis with network and host based data • BLN analysis with network data • Hybrid analysis with network and host based data • Hybrid analysis with network based data

28. Experimental Results

29. Future and Current Work • HoneyNet Project • Resource Management System with Intelligent System Processing at the Core

30. Conclusions • Intelligent System algorithms are very useful tools for applications in Network Security

31. Conclusions • Questions remain to be answered: • How will the system behave as the data becomes very noisy with respect to training data • How will other intelligence algorithms compare in performance—training time, accuracy, robustness in noise