Static Detection of Buffer Overflow in C using LCLint

1. Static Detection of Buffer Overflow in C using LCLint CS655 Group3

2. CENSORED Motivation RSA hacked ATN hacked One of the most common attacks is Buffer-Overflow-Attack.

3. The C language allows pointer arithmetic and array accesses without bounds checking, which makes writing data past the end of a buffer and overwriting other values possible Malicious users exploit this to cause programs to run arbitrary code --- classical buffer-overflow attack Buffer overflow examples: strcpy (char * arg1, char* arg2); /* bufSize (arg1) < stringLen (arg2) */ gets (char *s); /* length of stdin > sizeof (s) */

4. Two important reasons for buffer overflow to happen: 1. Non-Nullterminated strings are used where Null-terminated are expected; char * strcpy (char* dst, const char* src); 2. Size of storage buffer is smaller than the amount of data to be stored; strncpy (char *dst, const char* src, size_t n); /* buffer overflow happens if bufSize (dst) < n */

5. Related work on memory error detection: Dynamic approach: detect problems at run time, can only find problems that occur in the given test case, infeasible for applications constrained by space or time (real-time-systems) Example: Stackguard, GCCBOUNDS, PURIFY Static approach: analyze source code to find potential problems, can detect problems that are unlikely to occur in a test-case execution. Example: LCLint LCLint allows programmer-assisted-annotation /*@null@*/ char *x; /* x is null */

6. Our solution: modify LCLint to detect buffer overflows statically by introducing a series of annotations. nullterminated /*@nullterminated@*/ char *x; bufSize /*@bufSize=10@*/ char x[10]; stringLen /*@stringLen=11@*/ char *x=“Hello world”;

7. Formal semantics for the annotations Nullterminated (NT): 4 rules developed. Example: 3a. A |- x: NT A |- return x: ok 3b. A |- f_returnValue: NT, x: NT A |- return x: ok bufSize and StringLen: 12 rules developed

8. LCLint structure CHANGE TO SUPPORT NULLTERMINATED Lexical Analyzer Syntax checking Semantic analysis parser Complete Analysis

9. Implementation strategies of nullterminated : Modify the parser and introduce the checking for the “nullterminated” semantics in the analyzer; Associate the following information with each buffer: bbufstate bufstate = { NT, possibly NT, not NT}; /* state of the buffer */ int size; /* size of the buffer allocated */ int len; /* len of the buffer VALID ONLY if state is NT*/ The state of the buffer changes according to the inference made by the analyzer or the user-provided annotations; Both caller and callee of a function call need to honor error-checking requirements.

10. Evaluation *manually donemeans that the modified LCLint will be able to detect the buffer overflows after implementing bufSize and stringLen. We can manually trace down the problems now using these two annotations.

11. Example output: • Case 1: • Non-nullterminated passed as nullterminated. • Code: • #include <stdio.h> • int * temp (/*@nullterminated@*/ int * tptr) • { • char *x; • strcpy (x, tptr); • …. • return (NULL); • } • int * main () • { • int *c, *d; • c = malloc (34* sizeof (int) ); • temp(c); • d = c; • return d; • } Selected LCLint Messages: NULLTERMINATED ERROR test2.c: (in function main) test2.c:15:6: Possibly non-nullterminated storage c passed as nullterminated param: temp (c) A possibly non-nullterminated string/memory is used/referenced as a nullterminated one, (-nullterminated will suppress message) SYMBOL REFERENCED IS NOT NULLTERMINATED!

12. wu-ftpd: a convoluted, insecure real-world program /* / at start of path, set the start of the mapped_path to / */ if( path[0] == '/' ){ mapped_path[0] = '/'; mapped_path[1] = '\0'; path++; } while( (sl = strchr( path, '/' )) ){ char *dir, *last; dir = path; *sl = '\0'; path = sl + 1; if( *dir ) do_elem( dir ); if( *path == '\0' ) break; } if( *path ) do_elem( path ); if( (ret = chdir( mapped_path )) < 0 ){ strcpy( mapped_path, old_mapped_path ); } return ret; } /* From now on use the mapping version */ #define chdir(d) mapping_chdir(d) #define getwd(d) mapping_getwd(d) #define getcwd(d,u) mapping_getwd(d) #endif /* MAPPING_CHDIR */ char mapped_path[ MAXPATHLEN ] = "/"; /* Make these globals rather than local to mapping_chdir to avoid stack overflow */ char pathspace[ MAXPATHLEN ]; char old_mapped_path[ MAXPATHLEN ]; void do_elem(char *dir) { /* . */ if( dir[0] == '.' && dir[1] == '\0' ){ /* ignore it */ return; } } /* append the dir part with a leading / unless at root */ if( !(mapped_path[0] == '/' && mapped_path[1] == '\0') ) strcat( mapped_path, "/" ); strcat( mapped_path, dir ); } int mapping_chdir(char *orig_path) { int ret; char *sl, *path; strcpy( old_mapped_path, mapped_path ); path = &pathspace[0]; strcpy( path, orig_path );

13. A Real World Example /*@nullterminated@*/ int mapping_chdir( char * orig_path ) { /* ...*/ char *path; path = &pathspace[0]; strcpy(path, orig_path ); /* .. */ if( *path ) do_elem( path ); /*..*/ }

14. OFF BY ONE • char cdpath[MAXPATHLEN + 1]; • /*..*/ • snprintf(cdpath, sizeof cdpath, "%s/%s", ARG0, path); • if (chdir(cdpath) >= 0) { /*… */ } void do_elem( /*stringLen <= MAXPATHLEN *//*@nullterminated@*/ char *dir) { if( !(mapped_path[0] == '/' && mapped_path[1] == '\0') ) strcat( mapped_path, "/" ); strcat( mapped_path, dir ); }

15. Conclusions and future work We have already formally defined the semantics of some very important annotations “nullterminated”, “bufSize” and “stringLen” and have implemented and evaluated the “nullterminated” annotation as a proof of concept. Future work: Implementation of the “bufSize” and “stringLen” annotations and evaluation of their correctness. Annotations for handling possible buffer overflow errors in printf, scanf type functions could be added. Annotations that detect buffer overflow errors in variable argument lists could be added.

16. Acknowledgement Prof. Evans Yanlin Huang\0 Avneesh Saxena\0Seejo Sebastine\0 David Larochellepossibly not nullterminated! THANK YOU