Model Checking One Million Lines of C Code

1. Model Checking One Million Lines of C Code Hao Chen, UC Berkeley Drew Dean, SRI International David Wagner, UC Berkeley

2. MOPS (MOdel checking Programs for Security properties) • A static analysis tool that checks source programs for temporal safety properties.e.g. a setuid-root program must drop privilege before making risky system calls. • Analysis • Pushdown model checking • Inter-procedural • Control flow centric

3. The MOPS process Safety Property FSA Program satisifes safety property C Program CFG Model Checker Parser Error Traces FSA: finite state automaton CFG: control flow graph Treat the model checker as a black box for this talk

4. Is software model checking readyfor prime time? • Can model checking be used by open source developers to find security vulnerabilities? • Criteria for a successful tool • It is useful • Can check many properties • Can check diverse, widely-deployed programs • Requires moderate computational resources • It is usable • Can be used easily by non-tool developers • Can generate comprehensible error reports

5. Outline • Experiment • Programs: 8 widely-deployed programs, with over 1 million LOC • Properties: 5 security-related properties • Findings • More than a dozen vulnerabilities and weaknesses • Usability improvements • Conclusion

6. Programs

7. Security properties • Drop privilege completely when needed • Avoid stderr vulnerability • Avoid race condition (TOCTTOU) • Create chroot-jail safely • chdir(“/”) must follow chroot() immediately • Create temporary files safely • Use only the safe function mkstemp() • Never reuse filename in mkstemp(filename)

8. Property: drop privilege completely • Setuid-root programs should drop root privilege completely • before executing an untrusted program via system(), popen(), execvp() and friends, or • when the program intends to do so • Otherwise, the remaining privilege may be exploited by • the untrusted program that is executed • malicious code injected via buffer overrun attacks

9. Vulnerability: fail to drop privilege completely OpenSSH client (in readpass.c) seteuid(getuid()); setuid(getuid()); … execlp(askpass, askpass, msg, (char *) 0); …

10. What is wrong? OpenSSH 3.5 on OpenBSD OpenSSH 3.5 on Linux OpenSSH 2.5.2 on Linux R≠0, E=S=0 R≠0, E=S=0 R≠0, E=S=0 seteuid(getuid()) seteuid(getuid()) R=E≠0, S=0 R=E≠0, S=0 setuid(getuid()) setuid(getuid()) setuid(getuid()) R=E=S≠0 R=E≠0, S=0 R=E=S≠0

11. Potential Vulnerability • Weaknesses • ssh: fails to drop privilege before executing a user program • ssh-keysign: fails to drop privilege before doing complex cryptographic operations • A buffer overrun would allow the attacker to regain root privilege in euid.

12. Property: drop privilege completely

13. Vulnerability: stderr exploits in at Standard File Descriptors stdin stdout stderr Code tty tty tty tty <closed> <closed> close(1); close(2); attack.c tty <closed> <closed> execl(“at”, …); open(LFILE, O_WRONLY); tty LFILE <closed> at.c tty LFILE atfile fd = open(atfile, O_CREAT); Rule: No setuid-root program may open a file for writing to stderr

14. Property: stderr vulnerability

15. Summary of Findings

16. Outline • Experiment • Programs: 8 widely-deployed programs, with over 1 million LOC • Properties: 5 security-related properties • Findings • More than a dozen vulnerabilities and weaknesses • Usability improvements • Conclusion

17. Usability improvement 1:Make it really easy to run! • Problems • Packages have different build processes • Tool has to be manually configured for each package • Solution • Provide a script that integrates model checking into the build processes of packages automatically • Result: allow the user to run the tool as simple asmops –m setuid.fsa openssh-3.5p1-6.src.rpm

18. Integrating MOPS intoSoftware Build Processes • 1st attempt: manually edit Makefiles • Too complicated; does not survive autoconf • 2nd attempt: setenvGCC_EXEC_PREFIX to run MOPS instead of gcc • Build processes generate & run code • 3rd attempt: build CFG & machine code • Dangling CFGs; links to object files broken • 4th attempt: Put CFGs into ELF files • Solves all identified problems!

19. Usability improvement 2:report comprehensible error messages • Problem • One bug may trigger many error traces • The user has to review all the traces manually • Criteria for good error trace reporting • Reporting one error trace per bug • Reporting shortest error traces

20. Algorithm • Find the shortest error trace t and output it • Find the crucial statements on t, i.e. the first statement that causes an error on t • Prune s from the program • If the program still has error traces, go to step 1

21. Criteria for good tools: revisited • It is useful • Can check many properties • Can check diverse, widely-deployed programs • Requires moderate computational resources • It is usable • Can be used easily by non-tool developers • Can generate comprehensible error reports

22. Conclusion • Model checking is ready for prime time use by open source developers to find security vulnerabilities! • We believe that our experience would transfer to other similar tools as well. • Work in progress: check all 839 RPM packages in RedHat Linux 9