Efficient Context-Sensitive Intrusion Detection

1. Efficient Context-Sensitive Intrusion Detection J. Giffin, S, Jha and B. Miller CSCI 599 Presented by: Mikin Macwan

2. Host Based and Remote Intrusion Detection • Host Based Intrusion detection seeks to identify attempts to maliciously access machine on which the detection system executes • Remote Intrusion detection identifies hostile manipulation of processes running in a distributed computational grid.

3. Model based Intrusion detection • System under surveillance has a model of acceptable behavior for each monitored process. • Model is a description of the actions that a process is allowed to execute. • Unknown attacks and New attacks can be detected with few false alarms. • Number of false alarms are low or non-existent if the model is constructed right.

4. Model based Intrusion Detection • Can be constructed by • Human Specification and Training • Static source code analysis • Static binary code analysis

5. Dyck Model for Intrusion Detection • Dyck model uses static binary code analysis • No human intervention needed • No access to program source code needed • Model should contain all possible execution paths a process may follow • Prevent occurrence of false alarms

6. Program Analysis • Static Analysis • Context sensitive analysis • Develop a Push Down Automata (PDA) • PDA models processes most precisely • Higher precision results in lower efficiency • Context Insensitive Analysis • Develop context insensitive analysis models • Processes may not be modeled precisely • Model may include paths that originate from one function site but return to another call site – implies incorrect program execution • Dynamic Analysis • Constructs models from observed behavior during repeated training runs • Static analysis over approximates acceptable program behaviors and generates a model that may miss attacks • Dynamic analysis under approximates acceptable behaviors and leads to a high false alarm rate

7. Dyck Model • Efficient context sensitive program modeling • Previous work recommend using imprecise context insensitive models for reasonable performance • Dyck provides a precise context sensitive model with excellent performance characteristics • Model exposes call stack changes to the monitor • Monitor explores only the exact call path followed by the application • Null Call Squelching • Squelching reduces the generation of excessive null calls. • Does not compromise in security • Makes use of static and dynamic techniques to generate null calls that provide context for a system call • Data flow analysis to counter Mimicry attacks • Malicious code is camouflaged in a way that it behaves just like the application under consideration • Dyck model uses interprocedural data flow analysis to model arguments passed to and returned from system calls.

8. Dyck Model Construction • Binary Analyzer • Reads SPARC binary code and uses static analysis to construct program model • Additional rewrite to the binary code for more efficient modeling • User executes the rewritten binary in their security critical environment • Runtime Analyzer • Run time monitor tracks the execution of the binary to ensure that it follows the analyzer’s constructed model • Any deviation from the constructed model implies a security violation

9. C Code – Assembly Code Assembly code Original C Code System calls

10. Control Flow Graphs • C code converted to equivalent assembly code • Assembly code converted to corresponding Control Flow Graph (CFG). • Every function has a corresponding CFG • Combine all CFGs to create an Interprocedural model.

11. NFA Program Model • From previous slide, combine CFGs to a Non Deterministic Finite Automaton • Imprecise but efficient contest insensitive model • Suffers from impossible path exploits • Multiple different call sites to the same target procedure

12. PDA Program Model • Addition of Context sensitivity for greater precision • Call and return behavior of function calls is modeled using a PDA • Monitor traverses only matching call and return transitions, so no impossible paths exist in the system • Monitoring suffers from very high overhead values • Worst case complexity is cubic in the number of automaton states

13. Dyck Model • Efficient statically constructed context-sensitive model • Higher efficiency than the PDA due to limited state exploration • Null calls at selected function sites are inserted in the program • Precall – notifies the monitor of the calling function • Postcall – generated on return of the called function • If there is a Precall and Postcall mismatch, this indicates execution of an impossible path within the code

14. Selecting Instrumentation points • Recursive Functions • The model does not instrument recursive function calls • Recursive calls in the function are represented by Strongly Connected Components (SCC) • Each SCC is flattened into a single node • Call sites that do no execute a system call • Portions of the call graph are pruned which do not make any system calls • Monitor does not follow the program’s execution through such functions as no system call can be generated

15. Null Call Squelching • Squelching is used to eliminate redundant null calls • Null calls around a function call that returns without making a system call are discarded

16. Null Call Squelching • Separate squelch stack is created • Precall instrumentation pushes the call site identifier onto the squelch stack and nothing is sent to the identifier • Postcall code examines the state of the squelch stack • If stack is empty then a system call was made and all symbols were sent to the monitor • If the stack is not empty then no system calls were generated and the inserted precall is popped from the stack and no null calls are inserted

17. Evaluation • Precision • Means that the attacker has less chance inject malicious system calls • Precision based on average branching factor metric developed by Wagner and Dean • Branching factor is an indication of whether the attacker can inject dangerous system calls • Monitor records call potential dangerous system calls that could be called. Lower branching factor indicates lower chance of malicious system calls being made

18. Evaluation • Efficiency • Authors claim small runtime overhead in addition to existing process execution time • Measure this additional overhead in process execution • Measure each process’s runtime memory usage increase due to binary code instrumentation

19. Evaluation • Test Programs, workloads and statistics

20. Evaluation : Precision

21. Evaluation: Efficiency

22. Pros and Cons • Pros • Fast and Precise model • Cons • Do not know if sample test programs used are good enough to speak for larger and more complex applications

23. References • J.Giffin, S. Jha, B. Miller “Efficient Context-Sensitive Intrusion Detection” Computer Science Department, University of Wisconsin, Madison, WI • D. Wagner and D. Dean, “Intrusion Detection via Static Analysis. In IEEE Symposium on Security and Privacy, Oakland, California, May 2001.