Nozzle: A Defense Against Heap-spraying Code Injection Attacks

1. Nozzle: A Defense Against Heap-spraying Code Injection Attacks Paruj Ratanaworabhan, Benjamin Livshits, Benjamin Zorn – Microsoft Research Techical Report

2. Outline • Introduction • Nozzle design & implementation • Evaluation • False positives • False negatives • New threats (Adobe Reader) • Summary

5. Schematic of a Heap Spraying Attack

6. A typical JavaScript heap spray Allocates the shellcode Setting up the spraying NOP sled (256 KB) Allocates 1000 objects in browser’s heap (sled+shellcode)

8. Kittens of DoomWhat data can you trust? • Heap spraying is quite general, easy to implement • Many applications allow scripts in type safe languages • JavaScript, ActionScript • Java, C# • Many applications accept data from untrusted • Embed malicious code in images, documents, DLLs, etc. • Traditional signature based matching aren’t effective due to encoding and polymorphism

9. Nozzle System Architecture • Scanning objects locally • Maintaining heap healthy metrics globally

10. Local Malicious Object Detection • Is this object code? • Code and data look the same on x86 • Focus on sled detection • Majority of object is sled • Spraying scripts build simple sleds • Is this code a NOP sled? • Previous techniques do not look at heap • Many heap objects look like NOP sleds • 80% false positive rates using previous techniques • 80% of objects allocated by Mozilla Firefox look like sled code • Need stronger local techniques

11. Control Flow Graph (CFG) • Light-weight emulator scans heap objects to identify valid x86 code sequences • Disassembling the code and • Building a CFG • Focuses on detecting the NOP sled • Follows control flow to identify basic blocks that are likely to be reached through jumps from multiple offsets into the object

12. Object Surface Area Calculation (SA) • Assume: attacker wants to reach shell code from jump to any point in object • Goal: find blocks that are likely to be reached via control flow • Strategy: use dataflow analysis to compute “surface area” of each block

13. Object Surface Area Calculation • Each block starts with its own size as weight • Weights are propagated forward with flow • Invalid blocks don’t propagate • Iterate until a fix point is reached • Compute block with highest weight • SA: a proxy for the likelihood of control flow passing through the basic block

14. Global detection • To reduce the false positive rate of the local detection • Introduce global heap health metrics • Effectively distinguishes benign allocation behavior from malicious attacks • Determining whether a heap is under attack • Normalized attack surface area of the heap, NSA(H)

15. Nozzle Global Heap Metric

17. Summary of 10 benign web sites we used as Nozzle benchmarks • Measure false positives • Measure browser’s performance

20. Local false positive rate (without SA calculation)

21. Object surface area (for the misclassified objects from Nozzle)

23. Nozzle Runtime Overhead on a dual-core machine

24. Average error rate due to sampling

25. Case Study: Adobe Reader Exploit • Integer overflow error • Use Nozzle without any modification • Attack correctly detected • Attack Surface of the heap was greater than 94% • Sample rate 10% • Average CPU overhead 8% • Memory overhead 69% • Reduce by integrating Nozzle closely the underlaying memory manager • Confidence that Nozzle is capable of protecting a wide range of software

26. Summary • Heap spraying attacks are • Easy to implement, easy to retarget • In widespread use • Existing detection methods fail to classify malicious objects on x86 architecture • Nozzle • Effectively detects published attacks (known and new) • Has acceptable runtime overhead • Can be used both online and offline

27. Attacks on Nozzle • Injecting junk into start of object • Use multiple CNF at multiple starting offsets • TOCTTOU – When do you scan the object? • Scan objects once they mature • Future work • Periodically rescanning • Integrate with the garbage collector • Attacks on surface calculation • Jumps outside of objects • Multiple instances of shellcode inside an object lead to small SA • But attack failures lead to crashes • Sent message to browser vendors • Hiding the code itself • Code that rewrites heap at last minute

28. What about Data Execution Prevention • DEP / NX bit= hardware to prevent code execution on the heap • DEP is great, but isn’t used everywhere • Issues with app compatibility • DEP can be circumvented • JIT compilers complicate the story • Nozzle augments DEP for defense in depth