SCP: A System Call Protector against Buffer Overflow Attacks

1. SCP: A System Call Protector against Buffer Overflow Attacks 先進防禦實驗室 國立中央大學 資訊工程系

2. Outline • Introduction • Attacking Method • Related Work • SCP System Design • Experimental Result • Conclusion • Future Work

3. Introduction • Buffer Overflow Attack • Easily launched • Huge amount of targets • Strongly damage • One of the most dangerous threats in the Internet • Developing an efficient and effective approach becomes a critical and emergent issue.

4. Difficulty in Protection • Many countermeasures were proposed, but were defeated by new mutants.

5. Function and Stack Frame G(int a) { H(3); add_g: } H( int b) { char c[100]; int i; while((c[i++]=getch())!=EOF) { } } G’s stack frame b return address add_g H’s stack frame address of G’s frame point C[99] 0xabc Z Y X 0xabb Input String: xyz C[0] 0xaba

6. Stack Smashing Attack – Principle • Stack Smashing Attack • Overflow a return address to transfer program execution flow into injected code (shell code)

7. Stack Smashing Attack - Example Length=108 bytes G(int a) { H(3); add_g: } H( int b) { char c[100]; int i; while((c[i++]=getch())!=EOF) { } } Attack String: xxInjected Codexy0xabc b return address add_g addrress oxabc H’s stack frame address of G’s frame point y C[99] x Injected Code 0xabc 0xabb x x C[0] 0xaba

8. Return-into-libc Attack • Overwrite the return address to execute a library function, e.g. system(), inside the attacked process. High Address “/bin/sh” Pointer to system()’s arg Fake RA ESP system() EIP (RA) EBP buffer Low Address String format:

9. Heap Overflow Attack • Similar to stack overflow, but on the heap area

10. Other Mutants • function pointer attack • Jump table attack • setjmp/longjmp attack

11. Related Work • Some Countermeasures • StackGuard / StackShield • Address Obfuscation (ASLR/PaX) • Exec Shield • Binary Obfuscation • PointGuard™ • Instruction Set Randomization • RAD

12. High Address High Address return address return address canary word … Buffer … … Buffer Saved return address Low Address Related Work - StackGuard • StackGuard-- add canary word before return address • Bypassing StackGuard Low Address

13. Related Work – ASLR/PaX • Address Obfuscation • PaX/ASLR project - Randomize the base address of memory regions -- Randomize the base address of stack/heap -- Randomize the starting address of dynamic-linked libraries -- Randomize the locations of routines and static data • Internal fragmentation problem • Crash problem • Derandomization Attack • Local attack • Non-relocatable code

14. Related Work – Bound Check • Bound Check • Bound Check for C Program • Require source code / recompile • Runtime overhead are huge - 4x / 5x when best case - 10x general case - 100x worst case

15. Related Work – Exec Shield • Exec Shield • Data/Stack section non-executable • Code section non-writable • Compatibility problem - ELF file format -- Add PT_GNU_STACK and PT_GNU_HEAP - Nested function - Recompile / porting - sigreturn() system call • Return into libc attack can be launched

16. Related Work - PointGuard • PointGuard™ • Encrypt pointer, decrypt when reference object

17. Related Work - ISR • Instruction Set Randomization • Hardware solution, encrypt / decrypt CPU instructions • porting binaries

18. Related Work - RAD • RAD • Return Address Defender • Compiler solution • Push return address to RAR when prologue • Pop return address from RAR when eprologue • Need recompile

19. SCP System Principle and Goal • Principle • Prevent attackers from executing int 80 offered by them • Prevent attackers from executing int 80 existed in the attacked system • Goal • Low overhead • Efficient to protect • Do not require source code • Compatibility • Use less system resource (memory) • Easy to maintain

20. Assumption • Assumption • Malicious code have to use system calls to damage an attacked system • Vulnerable programs use dynamic linkedlibraries

21. System Call Invocation Convention movl system_call_number, %eax movl first_argument, %ebx … int $0x80

22. System Call Path with/without SCP

23. Detailed System Call Path without SCP High Address ( kernel ) sys_open() Kernel Space 5. return 4. trap into kernel ( libc wrapper routine ) __libc_open() User Space 3. call wrapper routine ( GOT entry ) Address of __libc_open 2. lookup GOT 6. return ( PLT ) jmp *GOT[__libc_open] 1. go to PLT ( user program ) open(); Low Address

24. Secure Enter Kernel • Pseudo Code (a) Secure Enter Kernel (b) Trap Code save_all_registers; page = 0; size = 0; if (page == 0 ) { page = mmap2(); size = copy_trap_code(page); notify_kernel(page+6); } restore_all_registers; call page; int $0x80; (sysenter) return_to_libc; machine code: \x8B\x44\x24\x04 \xCD\x80 \x83\xC4\x08 \xC3

25. Replacement • Use secure enter kernel to replace ALL sysenter (int $0x80)

26. Detailed System Call Path with SCP • New system call path (with SCP system) High Address ( kernel ) sys_open() 6. return Kernel Space sys_read() 5. trap into kernel ( trap page ) int $0x80 ( libc wrapper routine ) __libc_open() 4. call trap page User Space __libc_read() 7. return 3. call wrapper routine ( GOT entry ) Address of __libc_open Address of __libc_read 2. lookup GOT 8. return ( PLT ) jmp *GOT[__libc_open] jmp *GOT[__libc_read] 1. go to PLT ( user program ) open(); read(); Low Address

27. Lazy Binding under SCP glibc: printf() here 4. Call printf() 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

28. Lazy Binding under SCP glibc: printf() here 4. Call printf() 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

29. SCP Protection Mechanism – Limit the number of notify_kernel() • Avoid attacker’ using to change the address of the only legal int 80. • Limit the number of notify_kernel() to two: • loader (ld-linux.so.2) and glib (libc.so.6) have their own global variable segment; hence, there are two global variable pages in a process. • Each page variable will cause the system to create a new trap page.

30. SCP Protection Mechanism - Fake Trap Page • Insert fake trap pages around to real trap page to avoid attackers’ detection of the real int 80 instruction.

31. (user program) system_call (libc) wrapper routine (heap) int $0x80 (PLT) (GOT) Attack Analysis • Attacker can scan the attacked process’s address space to find the real int $0x80 • Possible solution: - Non-readable but executable code segment - … future work

32. Efficiency Test – Injected Code • Buffer overflow attack with injected code

33. Efficiency Test – Change Legal int 80 Address • Calling notify_kernel test

34. Performance Test – Micro Benchmark • 10,000,000 times per system call

35. Number of Page Faults under Original libc and Kernel avg timediff = 0.0000073216265842318534575253186069687672 sec = 7.32162658 usec Command exited with non-zero status 81 Command being timed: "./micro-test.open" User time (seconds): 0.69 System time (seconds): 122.31 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.00 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 82 Minor (reclaiming a frame) page faults: 7 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 81

36. Number of Page Faults under SCP libc and Kernel avg timediff = 0.0000123534624044317750067014868853298992 sec = 12.35346240 usec Command exited with non-zero status 82 Command being timed: "./micro-test.open" User time (seconds): 0.68 System time (seconds): 173.26 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:53.94 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 104 Minor (reclaiming a frame) page faults: 13 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 82

37. Performance Test – Micro Benchmark • 100,000 times per command

38. Conclusion • We propose a new method to protect system calls by registering valid int 80 on premise • Low performance overhead • Protect programs against all known code injection style BOAs • No modification to source code or executable files is required • Compatible with existing systems and applications • Not increase the workload of program maintenance • Terminate attacked processes normally • Effective in defending local BO attacks

39. Future Work • More secure • Implement “executable but non-readable” region in segment section on i386 • The NX Bit chip • AMD 64 CPU

40. Thanks • Q & A