Lecture 5. Heap & Function Pointer Overflow

1. COM850 Computer Hacking and Security Lecture 5. Heap & Function Pointer Overflow Han-Yee Kim Computer Science Education Korea University

2. Contents • Heap Overflow • Dynamic allocation • What is Heap? • A Basic Heap Based Overflow (Textbook) • Demo (1) with notetaker.c • Let’s change “datafile” • Demo (2) with notetaker.c • Function Pointer Overflow • What is function pointer? • Overflowing a Function Pointer (Textbook) • Let’s change player name • Demo (1) with game_of_chance.c

3. Heap Overflow

4. Dynamic allocation • Dynamic memory allocation is one of the memory allocation method which allocate memory space dynamically. (while executing a program.) • The dynamically managed memory space is maintained before garbage collection.(Programmer can also de-allocate that memory space.) From Wikepedia

5. What is Heap? Low addresses • Usually, memory is dynamically allocated from a large pool of unused memory area called the heap. (=free store) • Heap is One of the Data Structure for Dynamic memory allocation. The Heap grows down toward higher memory addresses. ↓ ↑ The Stack grows up toward lower memory addresses. High addresses Text Book Page(Page 75) From Wikepedia

6. What is Heap? Low addresses • This structure minimizes wasted space, allowing the stack to be larger if the heap is small and vice versa. The Heap grows down toward higher memory addresses. ↓ ↑ The Stack grows up toward lower memory addresses. High addresses Text Book Page(Page 75) From Wikepedia

7. What is Heap? Low addresses Simple example … Int *ptr; Ptr=Malloc(100); … The Heap grows down toward higher memory addresses. ↓ ↑ The Stack grows up toward lower memory addresses. High addresses Text Book Page(Page 75) From Wikepedia

8. A Basic Heap Based Overflow • If the writing data size is more than dynamically allocated memory space, What happen? void *ec_malloc(unsigned int size) { void *ptr; ptr = malloc(size); if(ptr == NULL) fatal("in ec_malloc() on memory allocation"); return ptr; }

9. A Basic Heap Based Overflow Low addresses • DEMO (100 Byte) (20 Byte) High addresses

10. A Basic Heap Based Overflow Low addresses • If the writing data size is more than allocated memory space (104byte), What happens? (100 Byte) (20 Byte) High addresses

11. A Basic Heap Based Overflow Low addresses • DEMO (100 Byte) (20 Byte) High addresses

12. A Basic Heap Based Overflow Low addresses • As predicted, when 104 bytes are tried, the null-termination byte overflows into the beginning of the “datafile” buffer. • This is a Basic Heap Based Overflow (100 Byte) (104 Byte) AAAAA… AAA… \0 (20 Byte) High addresses

13. Function Pointer Overflow

14. What is Function Pointer? Low addresses • A function pointer is a type of pointer in C, C++. and other C-like programming languages. • When dereferenced, a function pointer can be used to invoke a function and pass it arguments just like a normal function. Function {} *ptr High addresses From Wikepedia

15. Overflowing a Function Pointer • Stupid_Vault.c:A simple code made by me.(instead of Game_of_chance.c) • This program uses a function pointer for Vault’s state.

16. Overflowing a Function Pointer • An function pointer is stored in the main(). • The password buffer in the main() is a likely place for an overflow.

17. Overflowing a Function Pointer Low addresses • Initially, *lock_state points the locked_vault function. • If the password you typed is correct, *locked_state will point the open_vault function. void open_vault(); void locked_vault(); char password[10]; int(*lock_state) (); High addresses

18. Overflowing a Function Pointer • Demo with valid input size.

19. Overflowing a Function Pointer • Demo about address difference • Demo with invalid input size.

20. Overflowing a Function Pointer Low addresses • As like Demo, What happens if I write more than 28byte? • The pointer will point an wrong memory space! • This is an example of Function Pointer Overflow. void open_vault(); void locked_vault(); char password[10]; int (*lock_state) (); High addresses

21. Overflowing a Function Pointer Low addresses • Demo • Now open the vault with Function Pointer overflow! • Let’s type (anything 28byte+0x080483e4) void open_vault(); void locked_vault(); char password[10]; int(*lock_state) (); High addresses

22. Countermeasure the Attack • Demo • Let’s check the length of the string

23. Thank you!