120 likes | 346 Views
Compilers and Application Security. CSS 548 Dan Chock. Overview. What are some ways that compilers can affect application security ? Improving Application Security Checking for and preventing potential vulnerabilities Additional Effects on Security
E N D
Compilers and Application Security CSS 548 Dan Chock
Overview • What are some ways that compilers can affect application security? • Improving Application Security • Checking for and preventing potential vulnerabilities • Additional Effects on Security • Understanding compiler optimization for binary obfuscation • Reducing Application Security • Compiler vulnerabilities and backdoors
Improving Application Security • More Secure Code • Preventing stack buffer overflow exploitation via canaries “Anatomy of a Stack Smashing Attack and How GCC Prevents It”, Dr. Dobbs, Oliver Mueller, June 19, 2012
Improving Application Security • GCC Options • -fstack-protector; -fstack-protector-all • Adds a guard variable (aka canary) to functions that are initialized when a function is entered and checked when exited. If a guard check fails the program exits. • Visual Studio Switches • /GS • Shadowing to protect class member definitions, stack data reordering to protect function pointers
Improving Application Security • /sdl– Superset of /GS, expands scope and treats select security warnings as errors “Compiler Security Enhancements in Visual Studio 11”, SDL Team, MSDN, Dec. 2, 2011
Additional Effects On Security • Binary Obfuscation • volatilekeyword “Binary Obfuscation from the Top Down”, Sean Taylor
Binary Obfuscation • volatilekeyword “Binary Obfuscation from the Top Down”, Sean Taylor
Reducing Application Security • Compiler Vulnerabilities • Insecure Compiler Optimization • Improperly removing sensitive data from memory • Dead store removal - code that is used to overwrite memory is removed by the compiler • Compiler backdoors • Modifying compiler to recognize specific code while compiling and include a backdoor in the output • Ken Thompson - “Reflections on Trusting Trust” describes modified C compiler that would backdoor the login command to accept backdoor password
Specialized Compilers and Security • Compiler/hardware approaches to embedded systems security [1] • Secure compiler inserts hidden code into the compiled code that is validated during run-time by hardware • Security-preserving compiler for distributed programs [2] • Four stages: slicing, control flow protection (to keep track of state), replication (implement distributed program variables as local replicas), and cryptography (insert crypto operations to protect variable changes)
Additional References: • [1] “Performance Study of a Compiler/Hardware Approach to Embedded Systems Security”, Kripashankar Mohan, BhagiNarahari, Rahul Simha, Paul Ott1,Alok Choudhary, and Joe Zambreno • [2] “A Security-Preserving Compiler for Distributed Programs From Information-Flow Policies to Cryptographic Mechanisms”, CédricFournet, Gurvan Le Guernic, Tamara Rezk