Model Checking One Million Lines of C Code

Model Checking One Million Lines of C Code Hao Chen Drew Dean (SRI International) David Wagner with David Schultz, Geoff Morrison, Ben Schwarz Jacob West, Jeremy Lin

Outline • Overview of MOPS • Engineering effortsMake MOPS work on lots of real world applications • ExperienceSecurity properties and findings

MOPS (MOdel checking Programs for Security properties) • A static analysis tool that checks source programs for temporal safety properties • Main features • Pushdown model checking • Inter-procedural analysis • Control flow centric

Algorithm: Pushdown Model Checking • : set of security operations • e.g.  = { seteuid(!0), seteuid(0), execl()} • B*: sequences of security operations that violate the property • B is a regular language • The user describes B by an FSA • T*: sequences of security operationsfrom feasible traces in the program • T is a context-free language • MOPS automatically builds T from the program • Question: Is B  T = ? B T

The MOPS Process Safety Property FSA Program OK Program CFG Model Checker Parser Error Traces (they may violate the property) FSA: Finite State Automaton CFG: Control Flow Graph

Integrate MOPS into the Build Process • Goal: build a CFG for each target executable in the source package • Low-level programs • mops_cc1: parse each foo.c into foo.cfg • mops_ld: link all *.cfg together • How to use these programs to build CFGs for each package conveniently?

First Attempt: Edit Makefile • Feasible for packages with simple Makefiles • OpenSSH • Problems • Some Makefiles are automatically generated – they will be overwritten when the package is rebuilt • Too many Makefiles to edit – some packages generate one Makefile in each sub-directory • Some Makefiles are very complicated

Second Attempt: Interpose on the Build Process to Generate CFGs • Set the env variable GCC_EXEC_PREFIXto let gcc use • mops_cc1 for cc1 • mops_ld for ld • Therefore • Each source file foo.c is parsed into a CFG file foo.o • Each executable file a.out contains a linked CFG • Note that machine code is not generated

Second Attempt: Advantages and Problems • Advantage: works even if the build process • Moves files into other directories (mv) • Renames files (mv) • Creates libraries (ar) • Problem • Sometimes machine code is needed • By autoconf to test the functionality of the compiler • By the build process to generate C header files • But we let gcc create CFGs instead of machine code

Third AttemptBuild Both Machine Code and CFGs • Solution • Build both machine code and CFGs. For each foo.c, build both foo.o and foo.cfg • Problem • Machine code and its corresponding CFG may be separated if the build process • Moves and renames files (mv) • Creates and uses libraries (ar) • We need to “bind” machine code with its CFG

Final Attempt • Place both machine code and its CFG in the same object file • The ELF object format allows user-defined sections • For each source file foo.c • gcc’s cc1 generates machine code foo.o • mops_cc1 generates CFG foo.cfg • Run objcopy to place foo.cfg into foo.o

Now, as Easy as This • mops –m property.mfsa -- gcc foo.c bar.c • mops –b rpm –m property.mfsa -- rh9/*.src.rpm

Experiment: Checking Security Properties on Large Packages • Properties • Drop privilege before making unsafe system calls • Avoid race condition in file system access • Create root-jail safely • Create temporary files safely • Avoid stderr vulnerability • Programs • 7 programs, total one millions LOC

Property: Drop Privilege before Executing Untrusted Programs • Setuid-root programs need to drop root privilege before executing untrusted programs • exec...(), system(), popen() • Otherwise, the untrusted program will run with the root privilege

Property: Avoid Race Condition in File System Access • It is suspicious to make two system calls in a program path that use the same filename • E.g. • access(f) followed by open(f) • stat(f) followed by create(f) • Vulnerability • Symbolic links

Property: Create root jail securely • Rule: chroot() must be followed by chdir() immediately • A vulnerable program • The adversary sends in ../../etc/password from the network // cwd==/var/ftp chroot(“/var/ftp/pub”); filename = read_from_network(); fd = open(filename, O_RDONLY);

Property: Avoid Race Condition in Creating Temporary Files • victim.c:filename = mktemp(template);fd = open(filename, …); • But an adversary can create a file with the same name between the two statements • Then, victim.c will • Open the adversary’s file, or • Fail to create the temporary file (with the O_EXCL flag)

Rules for Avoiding Race Condition in Creating Temporary Files • Never use the unsafe functions:mktemp(), tmpnam(), tempname(), tmpfile() • Never reuse the parameter f in mkstemp(f) in any other function call, such as stat(), chmod(), etc • Because the same name may refer to a different file (tmpwatch)

Property: Avoid stderr Vulnerability int main() { // fd 0 and 1 are open, but fd 2 is closed … f = open(“/etc/passwd”, O_RDWR); // now /etc/passwd is opened to fd 2 fprintf(stderr, “%s”, user_input); // oops, have written to /etc/passwd }

Rule: Avoid stderr Vulnerability • Property • The first three file descriptors should not be opened by any file except /dev/null and /dev/zero

Property: stderr vulnerability

Property: create temporary files safely

Experiment: Build CFGs for All RPM Packages in Redhat 9.0 Linux • Number of packages: 840 • Time: 3 days 14 hours on 1.5G Pentium • Preliminary results: • Successful: 514 packages • Failed: 326 packages • Reasons for failure • Lack of prerequisite RPM packages(the major reason) • Bugs/limitations in MOPS’s parser • The package was not written in C

Summary • Engineering efforts in making MOPS work on lots of real world applications • Experience in checking security properties on large applications

Model Checking One Million Lines of C Code