KLIMAX: Profiling Memory Write Patterns to Detect Keystroke-Harvesting Malware

1. 14th International Symposium on Recent Advances in Intrusion Detection (RAID 2011) KLIMAX: Profiling Memory Write Patterns to Detect Keystroke-Harvesting Malware Stefano Ortolani1, Cristiano Giuffrida1, and Bruno Crispo2 1VrijeUniversiteit 2University of Trento 左昌國 Seminar @ ADLab, NCU-CSIE

2. Outline • Introduction • Approach • Optimization • Evaluation • Discussion • Related Work • Conclusions

3. Introduction • Keylogger detection • Signature-based solutions • Evasion techniques • Signature producing time • Behavior-based solutions (system calls or library calls invoking) • False positives • False negatives

4. Introduction • This paper proposes a new behavior-based detection model • KLIMAX : Kernel-Level Infrastructure for Memory And eXecution profiling • Based on memory write pattern profiling • Proactive and Reactive • Previous work • Stefano Ortolani, Cristiano Giuffrida, and Bruno Crispo, Bait your Hook: a Novel Detection Technique for Keyloggers, RAID 2010 • Comparing I/O patterns • FN: by delaying or disguising I/O activities

5. Approach • To ascertain the correlation between the stream of issued keystrokes and the memory writes a process exhibits. • High correlation means keylogging behaviors exist. • No virtualization techniques • Kernel-level solution • Does not provide kernel rootkit detection

6. Approach

7. Approach • Detector • The detector uses the statistical suite R to randomly generate patterns • Write patterns received from the Injector • Categorized: data, stack, heap • Computing the correlation between 2 patterns • PCC: Pearson product-moment Correlation Coefficient • Injector • A virtual keyboard driver • Converting patterns into keystroke streams

8. Approach • IDT – Interrupt Descriptor Table • ref

9. Approach

10. Approach • Shadower • Classifier

11. Approach

12. Optimization • To reduce the false positives and false negatives • Many benign applications would register callback functions to intercept keystroke event • High correlation • The callback mechanism is implemented in USER32.dll • Transient memory write patterns on stacks at callback execution time(short-lived stack)  avoid logging any memory writes performed by USER32.dll • Identifying long-lived regions of the stack during execution • Excluding any other stack region • Adaptive algorithm to identify long-lived stack • Initially, marking entire stack as long-lived stack • As the execution progresses, sampling the stack pointer of each thread at regular time intervals and update the deepest value.

13. Evaluation • Synthetic Evaluation

14. Evaluation

15. Evaluation • False Positive Analysis • Static binary analysis(or dynamic analysis) • Standard API • SetWindowsHookEx, GetKeyState, GetAsyncKeyState( from USER32.dll) • Hotkey registration API • RegisterHotKey

16. Discussion • The main strength of the detection strategy is to detect keylogging behavior within short windows of observation even for malware buffering data for a long time. • False Positives • If a benign application keeps sensitive data in global memory regions  this is unnecessary behavior • In the False Negative evaluation • 2 samples represent that proactive method is not a good idea • Event trigger based “reactive” should be good

17. Related Work • Behavior-based approach (malware detection) • Polymorphic malicious executable scanner by api sequence analysis • Malware profiling • Behavior-based spyware detection • Effective and efficient malware detection at the end host • API correlation • Detecting bots based on keyloggingactivities • Bait your hook: a novel detection technique for keyloggers

18. Conclusions • KLIMAX: a kernel-level infrastructure to analyze and detect malware with generic keylogging behavior • Can be deployed on unmodified Windows-based systems • Proactive detection • No false positives • No false negatives (the keyloggingbahavior is triggered within the window of observation) • Reactive detection • Policy-based reactive detection • No false negatives in “general” case • Antivirus misclassified several malware