Nozzle: A Defense Against Heap Spraying Attacks

Nozzle: A Defense Against Heap Spraying Attacks Ben Livshits Paruj Ratanaworabhan Ben Zorn

A Brief History of Memory Exploits Stack overflow Heap exploit Heap spraying Frequency Vista heap,layout randomization Nozzle StackGuard 2000 2002 2004 2006 2008 2010 Year

Stack Overflow Exploit 1 2 exploit jump stack return address nop sled shellcode <IFRAME SRC=file://BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB … NAME="CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC … ഍഍"></IFRAME>

Heap Corruption Exploit Heap 1 2 exploit jump vtablepointer nop sled <IFRAME SRC=file://BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB … NAME="CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC … ഍഍"></IFRAME> shellcode

Heap Spraying Exploit Heap 2 1 3 spray jump exploit vtablepointer sled sled sled sled sled sled sled sled sled sled sled <IFRAME SRC=file://BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB … NAME="CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC … ഍഍"></IFRAME> <SCRIPT language="text/javascript"> shellcode= unescape("%u4343%u4343%...''); oneblock= unescape("%u0C0C%u0C0C"); varfullblock = oneblock; while(fullblock.length<0x40000) { fullblock+= fullblock; } sprayContainer= new Array(); for(i=0; i<1000; i++) { sprayContainer[i] = fullblock + shellcode; } </SCRIPT> shellcode shellcode shellcode shellcode shellcode shellcode shellcode shellcode shellcode shellcode shellcode

Kittens of Doom. Is no data sacred? • Spraying: general attack • Embed malicious code in images, documents, DLLs, etc. • Image example: • Comments • Transformed data • Documented at BH’08

Heap Spraying is a Real Problem • Drive-by exploits • Just visiting a site can compromise your whole machine

Nozzle Overview Heap Spraying Nozzle Detect / mitigate heap spray attack Monitors heap for suspicious activity Compare to HW “no-execute” page protection More compatible Doesn’t just crash Focus on browser, but applicable to all applications • Relies on pre-existing exploit (in C/C++) • Spraying in type-safe language • JavaScript, C#, Java • JIT-ed languages: good targets • Randomization doesn’t help • Browsers are popular target

Nozzle Architecture

Nozzle Experimental Summary

Local vs. Global Detection 000000000000000000000000000000000000000000000000000000000000000000000000 000000000000 add [eax], al add [eax], al add [eax], al add [eax], al add [eax], al add [eax], al add [eax], al Code or Data? Local Detection: Is this object dangerous? Code and data: same on x86 Local detection: 80% FP rate Nozzle: collections of objects Sprayed heap: large attack surface 0101010101 0101010101 0101010101 0101010101 0101010101 0101010101 0101010101 and ah, [edx] and ah, [edx] and ah, [edx] and ah, [edx] and ah, [edx] and ah, [edx] and ah, [edx] Global Detection: Is my heap under attack? 11

Nozzle Global Heap Metric Normalize to (approx): P(jump will cause exploit) build CFG Compute threat of entire heap dataflow Compute threat of single block Compute threat of single object

Attack Surface Calculation sub [ eax ] , eax Legend : adc dh , bh arithmetic or eax , 0 d 172004 h memory I / O or syscall in eax , 0 x 11 control flow test cl , ah jecxz 021 c 7 fd 8 add [ eax ] , al add al , 30 h add [ ecx ] , 0 add al , 80 h add al , 38 h outs dx , [ esi ] jecxz 021 c 7 fde xor [ eax ] , eax k c o imul eax , [ eax ] , 6 ch l b t e or eax , 0 d 179004 h g r a t o T A B C D An example object from visiting google.com • Extract control flow graph (CFG) from heap object • SA(Bi) = likelihood of ending in Biif we land within object boundaries • A BB contributes its effective size to another BB’s SA, if there is a path to that other BB • BB containing prohibitive instructions has zero effective size • int, out, hlt, or ltr 13

economist.com versus mw-612 (actual attack) Normalize Surface Area Logical time (number of allocations/frees)

Nozzle Runtime Overhead 20% 10% 4X 50% 15

Summary • Heap spraying is a real threat to Windows, Office, … • Can be launched with JavaScript, C#, Java, Images, mp3s,… • Code/data is difficult to distinguish • Published approaches fail • Heap spraying affects global heap health, • Detected by Nozzle • Effectively identifies spraying at low cost • Product groups have already expressed interest

Future Work • Closely integrate Nozzle with • IE • .NET/CLR (garbage-collected heap vulnerable too) • Improve filtering, anticipate new attacks • Mitigate after detecting • Address TOCTOU with GC and/or identifying stores to heap

Nozzle: A Defense Against Heap Spraying Attacks Ben Livshits Paruj Ratanaworabhan Ben Zorn

False positive results Maximum normalized SA for Alexa top 150 (top) and 10 selected sites (bottom) No more than 12% of max SA reported No false positives reported for 20% threshold What about SA for rogue sites? 19

Backup: SA for various benign sites 20

Nozzle versus DEP But, Nozzle is more compatible and more selective DEP prevents code execution in memory Can be disabled at runtime Has compatibility issue Circumvented with Java (Applet) 21

False negative results Maximum normalize SA Over 8 times of the max of the benign sites Over 6 times of the max of the benign sites • 12 published heap spray pages • 2000 synthetic heap spray pages • Use MetaSploit’s advance NOP engine and shellcode database 22

Effect of sampling on max SA calculation Average error rate Test with the 10 selected sites Measure the error with respect to 100% 23

Nozzle: A Defense Against Heap Spraying Attacks