Security Vulnerability Analysis and Mitigation for Real-World Systems

1. Security Vulnerability Analysis and Mitigation for Real-World Systems Shuo Chen Center for Reliable and High-Performance Computing Coordinated Science Laboratory University of Illinois at Urbana-Champaign Final Exam, August 18th, 2005 Committee Chair: Prof. Ravi Iyer Committee: Prof. Vikram Adve Prof. Ravi Iyer Prof. Jose Meseguer Prof. David Nicol

2. Prelim Exam Recap • Analyzed security vulnerability reports in Bugtraq and CERT advisories • Most vulnerabilities can be modeled as a series of simple logic predicates. • Used FSM models to reason about many categories of vulnerabilities. • A common characteristic of most security vulnerabilities: pointer taintedness • Pointer value derived from user input • Allow users to specify memory addresses. Usually due to attacks! • Developed a theorem proving approach to reason about possibility of pointer taintedness • To uncover potential vulnerabilities.

3. Since Prelim Exam • Questions focused • Is pointer taintedness detection just an alternative approach to existing defense techniques, or is it a significant improvement? • Is pointer taintedness detection applicable to large real-world software?

4. Since Prelim Exam (cont.) • Contributions • Demonstrate that a new security attack – non-control-data attack, is applicable to many real-world software, not addressed by many current defense techniques. • Demonstrate that pointer taintedness detection can naturally defeat non-control-data attacks as well as traditional attacks. • Demonstrate that pointer taintedness detection can be deployed in large systems. • By building into processor architecture • By combining theorem proving and runtime assertions

5. Summary of My Research • Start from the analysis of a large volume of security data • Extract common characteristics of security vulnerabilities and attacks • Propose new defense techniques (supported by real-world attack models)

6. Publications • S. Chen, J. Xu, E. C. Sezer, P. Gauriar and R. K. Iyer. "Non-Control-Data Attacks Are Realistic Threats," USENIX Security Symposium, 2005. • S. Chen, J. Xu, N. Nakka, Z. Kalbarczyk, R. K. Iyer. “Defeating Memory Corruption Attacks via Pointer Taintedness Detection,” DSN, 2005. • S. Chen, J. Dunagan, C. Verbowski and Y.-M. Wang, “A Black-Box Tracing Technique to Identify Causes of Least-Privilege Incompatibilities,” NDSS, 2005. • S. Chen, J. Xu, Z. Kalbarczyk, R. K. Iyer. “Security Vulnerabilities: From Analysis to Detection and Masking Techniques,” Proceedings of the IEEE, 2005. • S. Chen, K. Pattabiraman, Z. Kalbarczyk, R. K. Iyer, "Formal Reasoning of Various Categories of Widely Exploited Security Vulnerabilities Using Pointer Taintedness Semantics," IFIP SEC, 2004 • S. Chen, J. Xu, Z. Kalbarczyk, R. K. Iyer and K. Whisnant. “Modeling and Evaluating the Security Threats of Transient Errors in Firewall Software,” Performance Evaluation, 2004. • S. Chen, Z. Kalbarczyk, J. Xu, R. K. Iyer. "A Data-Driven Finite State Machine Model for Analyzing Security Vulnerabilities," DSN, 2003. • S. Chen, J. Xu, R. K. Iyer, K. Whisnant. "Modeling and Analyzing the Security Threat of Firewall Data Corruption Caused by Instruction Transient Errors," DSN, 2002. • J. Xu, S. Chen, Z. Kalbarczyk, R. K. Iyer. "An Experimental Study of Security Vulnerabilities Caused by Errors," DSN, 2001. 9 full papers in IEEE DSN, USENIX Security, IFIP Security, ISOC NDSS, Proceedings of IEEE and Journal of Performance Evaluation

7. Non-Control-Data Attacks Are Realistic Threats (Joint work with Jun Xu) In USENIX Security Symposium, 2005

8. Control Data Attack: Well-Known, Dominant • Control data attack: corrupt function pointers, jump targets and return addresses to run malicious code • Currently the most dominant form of memory corruption attacks [CERT and Microsoft Security Bulletin] • By exploiting many vulnerabilities such as buffer overflow, format string bug, integer overflow, double free, etc. • Many current defense techniques: to enforce control data integrity to provide security. • Monitor system call sequences (Intrusion detection systems) • Protect control data (Secure Program Execution, Minos) • Non-executable stack and heap (Linux, OpenBSD, Windows XP SP2)

9. Non-Control-Data Attack • Non-control-data attacks: attacks not corrupting any control flow data • Currently very rare in reality • Very few instances documented in literature. • Several papers: possible to construct non-control-data attack against synthetic programs. • Not yet considered as a serious threat • How applicable are such attacks against real-worldsoftware? • Why rare  attackers’ incapability or lack of incentives? • No focused investigation yet.

10. Our Claim: General Applicability of Non-Control-Data Attacks • The claim: • Many real-world software applications are susceptible to non-control-data attacks. • The severity of the attack consequence is equivalent to that due to control data attacks. • Goal of our project • Experimentally validate the claim • Construct non-control-data attacks to compromise the security of widely-used applications • Discuss limitations of current defense techniques • Show that pointer taintedness detection can defeat both control-data attacks and non-control-data attacks.

11. x uninitialized, run as EUID 0 x=109, run as EUID 0 x=109, run as EUID 109. Lose the root privilege! Get a special SITE EXEC command. Exploit a format string vulnerability. x= 0, still run as EUID 109. Get a data command (e.g., PUT) x=0, run as EUID 0 x=0, run as EUID 0 Non-Control-Data Attack against WU-FTP Server (via a format string bug) int x; FTP_service(...) { authenticate(); x = user ID of the authenticated user; seteuid(x); while (1) { get_FTP_command(...); if (a data command?) getdatasock(...); } } getdatasock( ... ) { seteuid(0); setsockopt( ... ); seteuid(x); } When return to service loop, still runs as EUID 0 (root). Allow me to upload /etc/passwd I can grant myself the root privilege! Only corrupt an integer, not a control data attack.

12. Non-Control-Data Attack against NULL-HTTP Server (via a heap overflow bug) • Attack the configuration string of CGI-BIN path. • Mechanism of CGI • suppose server name = www.foo.comCGI-BIN = • Requested URL = http://www.foo.com/cgi-bin • The server executes • Our attack • Exploit the vulnerability to overwrite CGI-BIN to /bin • Request URL http://www.foo.com/cgi-bin/sh • The server executes /usr/local/httpd/exe /usr/local/httpd/exe /bar /bar /bin /sh The server gives me a root shell! Only overwrite four characters in the CGI-BIN string.

13. auth = 0 auth = 0 auth = 1 auth = 1 Password incorrect, but auth = 1 Logged in without correct password Non-Control-Data Attack againstSSH Communications SSH Server (via an integer overflow bug) void do_authentication(char *user, ...) { int auth = 0; ... while (!auth) { /* Get a packet from the client */ type = packet_read(); switch (type) { ... case SSH_CMSG_AUTH_PASSWORD: if (auth_password(user, password)) auth =1; case ... } if (auth) break; } /* Perform session preparation. */ do_authenticated(…); }

14. More Non-Control-Data Attacks • Against NetKit Telnet server (default Telnet server of Redhat Linux) • Exploit a heap overflow bug • Overwrite two strings:/bin/login –h foo.com -p (normal scenario) /bin/sh –h –p -p (attack scenario) • The server runs /bin/sh when it tries to authenticate the user. • Against GazTek HTTP server • Exploit a stack buffer overflow bug • Send a legitimate URL http://www.foo.com/cgi-bin/bar • The server checks that “/..” is not embedded in the URL • Exploit the bug to change the URL to http://www.foo.com/cgi-bin/../../../../bin/sh • The server executes /bin/sh

15. What Non-Control-Data Attacks Imply? • Control data integrity is not sufficient to ensure software security for real-world software. • Many types of non-control data critical to security • User identity data, configuration data, user input text string and decision-making Boolean • Once attackers have the incentive, they are likely to succeed in non-control-data attacks.

16. Runtime Pointer Taintedness Detection at Processor Level Joint work with Jun Xu and Nithin Nakka In IEEE International Conference on Dependable Systems and Networks (DSN), 2005

17. Recap: Pointer Taintedness • The root cause of many memory corruption attacks: pointer taintedness • No matter whether they overwrite control-data or non-control-data • Many type of vulnerabilities: e.g., buffer overflow, format string, heap corruption, integer overflow, and globbing attacks. • Pointer taintedness: a pointer value is derived from user input • In prelim, I showed a theorem proving technique to reason about possibility of pointer taintedness

18. \x20 \xbc \x02 \x10 %d %d %d %n fmt: format string pointer ap: argument pointer fmt: format string pointer ap: argument pointer Is a Format String Attack Due to Pointer Taintedness? Vulnerable code: recv(socket,filename); sprintf(buf,”%s not found”,filename); printf(buf); /* should be printf(“%s”,buf) */ Suppose user ID, CGI-BIN or critical flag in 0x1002bc20 High … %n %d %d %d 0x1002bc20 Stack growth Low In vfprintf(), if (fmt points to “%n”) then **ap = (character count) *ap is the tainted value 0x1002bc20.

19. Runtime Pointer Taintedness Detection • A processor architectural level mechanism to detect pointer taintedness • On SimpleScalar processor simulator • Implemented a taintedness-aware memory system • One-bit extension for each byte, similar to the parity bit, to indicate the taintedness of this byte • Taintedness tracking • Taintedness is propagated by ALU instructions • Taintedness initialization • read and recv system calls: tag every byte of receiving buffer as tainted • Attack detection • When a tainted value is dereferenced (i.e., used as a pointer).

20. ALU taintedness tracking logic Data pointer taintedness detector 36 bits 36 bits 36 bits 36 bits 36 bits 36 bits 36 bits Jump pointertaintedness detector ID/EX MEM/WB EX/MEM Opcode store path Compare specific logic Shift specific logic AND specific logic XOR specific logic 4 bits 4 bits Bitwise OR 4 bits Register File M U X alert MUX Data Memory MUX load/store? jr? M U X M U X alert 0 0 4 bits M U X 36 bits 32 bits A L U 32 bits 8-bit byte 32 bits Taintedness bit load path 36 bits

21. Evaluation • Effectiveness of attack detection • Synthetic vulnerable programs • Real-world network applications • Evaluation of false positives • Real-world network applications • SPEC 2000 benchmarks • Potential false negative scenarios • A few attack scenarios that are not detected

22. Effectiveness of Attack Detection • First, test on synthetic vulnerable programs • All attacks are detected and terminated

23. Attack Detection Effectiveness (cont.) • Test on real network applications • All attacks are detected • No difference between control-data attack and non-control-data attack from the viewpoint of pointer taintedness

24. Evaluation of Transparency and False Positives • Transparent: precompiled binary executables can run • Test on network applications • No attack  no alert • Test on SPEC benchmarks • Execute 15,139 million instructions without any alert • Conclusion: No known false positive

25. Potential False Negative Scenarios • Incorrect array index boundary check • Determining correct array size requires source code analysis – very hard at binary level • Buffer overflow within the local frame • If no pointer is tainted, no alert is raised • Unlikely to cause severe security damage because attacker-controllable location is very limited • Format string attack causing information leak • This attack allows peeking a few words on the top of the stack. • Cause security compromises if these words contain security-critical secret, e.g., key and password

26. Combining Static Analysis and Runtime Detection

27. Towards An Easier Deployment of Pointer Taintedness Detection • Advantage/limitation of static analysis • to derive assertions (when satisfied, eliminate pointer taintedness) • No need for hardware modification, but not easy to deploy in large programs • Advantage/limitation of runtime detection • Easy to deploy in large programs, but needs modification of the processor • Can we combine the two? • Static analysis to extract security specifications of critical functions • Enforce these specifications by runtime assertions • Purely a software approach (of course, we can also design hardware to enforce runtime assertions)

28. VC(1): the specification (^a=1 => T(^^p)= false)  (^a≠1 => T(^p)= false) compile 1: branch (~(a is 1)) 3 2: mov [p] <- ^p + 1 3: mov [q] <- ^p - 2 4: mov [^q] <- 12 1: branch (~(^a is 1)) go 3 2: mov [p] <- ^^p + 10 3: mov [q] <- ^p - 2 4: mov [^q] <- 12 VC(2): T(^^p)=false VC(3): T(^p)=false VC(4): T(^q)=false Verification Condition (VC) Generation char *p, *q; if (a == 1) p = *p + 10; q = p - 2; *q = 12;

29. Case Study: free() typedef struct _HEAP_BLOCK { int Size; int Busy; struct _HEAP_BLOCK * Fwd,* Bak; } HEAP_BLOCK, * PHEAP_BLOCK; char * BlockSizes; void free(char * p) { int BlockSize,i; char * BuddyBlock,* FreedBlock; int FreeBlockListIndex,MergeExit; FreedBlock=p-sizeof(HEAP_BLOCK); // Mark this block free. FreedBlock->Busy=0; BlockSize=FreedBlock->Size; FreeBlockListIndex = CalculateFreeBlockListIndex(BlockSize); FreeBlockListIndex=0; while (BlockSize > *(BlockSizes+FreeBlockListIndex)) { BlockSize = BlockSize / 2; FreeBlockListIndex++; } MergeExit=0; while (FreeBlockListIndex < 6 && MergeExit==0) { BuddyBlock = HEAP_BASE + (FreedBlock- HEAP_BASE) ^ BlockSize; if (BuddyBlock->Busy || BuddyBlock->Size != BlockSize) MergeExit=1; else { // Make a bigger block and free it. BlockSize*=2; FreeBlockListIndex++; if (BuddyBlock<FreedBlock) FreedBlock = BuddyBlock; BuddyBlock->Fwd->Bak=BuddyBlock->Bak; BuddyBlock->Bak->Fwd=BuddyBlock->Fwd; } } FreedBlock->Size = BlockSize; \ FreedBlock->Busy = 0; InsertTailList(FreeBlockListIndex, FreedBlock); } Compile inst(1) = mov [FreedBlock] <- (^ p - 16) . inst(2) = mov [^ FreedBlock + 4] <- 0 . inst(3) = mov [BlockSize] <- ^ ((^ FreedBlock + 0)) . inst(4) = mov [FreeBlockListIndex] <- 0 . inst(5) = no-op . inst(6) = branch (~(^ ((^ BlockSizes + ^ FreeBlockListIndex)) < ^ BlockSize)) 10 . inst(7) = mov [BlockSize] <- (^ BlockSize / 2) . inst(8) = mov [FreeBlockListIndex] <- (^ FreeBlockListIndex) + 1 . inst(9) = branch true 5 . inst(10) = no-op . inst(11) = mov [MergeExit] <- 0 . inst(12) = no-op . inst(13) = branch (~(^ FreeBlockListIndex < 6 && ^ MergeExit is 0)) 28 . inst(14) = mov [BuddyBlock] <- ((HEAP_BASE + ((((^ FreedBlock - HEAP_BASE)) xor ^ BlockSize)))) . inst(15) = branch (~(~(^ ((^ BuddyBlock + 4)) is 0) || ~(^ ((^ BuddyBlock + 0)) is ^ BlockSize))) 18. inst(16) = mov [MergeExit] <- 1 . inst(17) = branch true 26 . inst(18) = no-op . inst(19) = mov [BlockSize] <- 2 . inst(20) = mov [FreeBlockListIndex] <- (^ FreeBlockListIndex) + 1 . inst(21) = branch (~(^ BuddyBlock < ^ FreedBlock)) 23 . inst(22) = mov [FreedBlock] <- ^ BuddyBlock . inst(23) = no-op . inst(24) = mov [^(^ BuddyBlock + 8) + 12] <- ^ (^ BuddyBlock + 12) . inst(25) = mov [^(^ BuddyBlock + 12) + 8] <- ^ (^ BuddyBlock + 8) . inst(26) = no-op . inst(27) = branch true 12 . inst(28) = no-op . inst(29) = mov [^ FreedBlock + 0] <- ^ BlockSize . inst(30) = mov [^ FreedBlock + 4] <- 0 . inst(31) = no-op . VC generation VC(1): T (^ p) = false  T (^ (^ x + 8)) = false  T (^ (^ x + 12)) = false x = (((p-16) - HEAP_BASE) xor ^(p-16)) + HEAP_BASE

30. Case Study: free() (cont.) void free(char * p) { HEAP_BLOCK * x=(HEAP_BLOCK*) (HEAP_BASE + (((p-16) - HEAP_BASE) ^ (*(UINT*)(p-16)))); assert (x->Fwd->Bak == x && x->Bak->Fwd == x); … … … … ( the original source code of free() ) } • Runtime enforcement of VC using a runtime assertion • Effectiveness /* try to hijack *f() to buffer p */ int main() { char * p; void (*f)(); p = malloc(40); *(UINT*)(p+60)=(UINT)p; *(UINT*)(p+56)=((UINT)&f)-12; free(p); } Heap corruption attack. Assertion is violated!

31. Case Study: vfprintf() while (n>0) { n--; done++; } } else if (*p==’s’) { q=*ap; if (q==0) break; while (*q!=0) { done++; q++; } } else if (*p==’n’) { q = *ap; *(int *) q = done; done++; } else { done++; } state=1; } p++; } return done; } int vfprintf (char *s, char *format, char * ap) { char * p, *q; int done,state,data,n; char buf[10]; p=format; done=0; if (p==0) return 0; state=1; while (*p != 0) { if (state==1) { if (*p==’%’) state=0; else done++; } else { if (*p==’%’) { done++; } else if (*p==’d’) { data=*ap; if (data<0) { done++; data=-data; } n=0; while (data>0 && n<10) { *(&buf+n)=data%10+’0’; data/=10; n++; }

32. Case Study: vfprintf() (cont.) VC(8) = (~ (^ state = 1) && ^ ^ p = ‘n’) -> (T(^ ap) = false) • Extracted VC • Runtime enforcement of VC using a runtime assertion int vfpintf (FILE *s, const char *format, va_list ap) { … while (*p != 0) { assert (!(state != 1 && *p==‘n’ && !UNTAINTED(ap))); } } int printf (const char *format, ...) { return vfprintf (stdout, format, arg); } void main() { mov %esp, stack_top; ADD_UNTAINTED_ADDR (stack_top-4); printf("string=%s\ni=%d\n%n",buf,i,&j);REMOVE_UNTAINTED_ADDR (stack_top-4); scanf(“%s”,buf); printf(buf); } Legitimate call. Assertion holds Format string attack. Assertion is violated

33. Conclusions

34. Conclusions • Most security vulnerabilities (in Bugtraq and CERT) can be modeled as a series of violations of logic predicates • Promising to apply formal method to analyze software security (shown in prelim exam) • Many real-world software can be compromised by corrupting non-control data. • Need a more comprehensive defense technique • Pointer taintedness is a unifying perspective to reason about most memory corruption vulnerabilities/attacks. • Effective for defeating both control-data attacks and non-control-data attacks • Detecting about pointer taintedness is a promising direction to enhance security on real-world systems • Techniques explored: • theorem proving (shown in prelim exam) • runtime detection • combination of automatic VC generation and runtime assertion