CSC 382: Computer Security

1. CSC 382: Computer Security Software Security Testing CSC 382: Computer Security

2. Topics • Why is Testing Important? • White Box vs Black Box Testing • Creating a Test Plan • Fault Injection • Source-based Fault Injection • Environment Fault Injection • Input-based Fault Injection • Fuzz Testing CSC 382: Computer Security

3. Why is Testing Important? Navy shoots down Civilian Airliner. • 1988 • US Vicennes shot down Airbus 320. • Mistook airbus 320 for a F-14. • 290 lives lost. • Why: Software bug – “cryptic and misleading output displayed by the tracking software.” CSC 382: Computer Security

4. Why is Testing Important? “I Love You” virus cost $8.75 billion.2 • Result of Microsoft decision to automatically execute binary attachments in Outlook. CSC 382: Computer Security

5. Why is Testing Important? • Complexity • Continually increasing. • Windows 3.1 (3mloc) to Windows XP (40mloc) • Extensibility • Plugins. • Mobile code. • Connectivity • Network access. • Wireless networking. CSC 382: Computer Security

6. Software Complexity 5-50 bugs per/kloc8 • 5/kloc: rigorous quality assurance testing (QA) • 50/kloc: typical feature testing CSC 382: Computer Security

7. Software Connectivity • Internet connectivity nearly ubiquitous. • 802.11 wireless networking. • Bluetooth wireless personal area networking. • Embedded devices often networked. • Can your cellphone get a virus? • Symbian Bluetooth virus released in 2004. • Can your automobile get a virus? • BT discovery reveals embedded BT devices in high-end cars. CSC 382: Computer Security

8. Testing Caveat "Testing can establish the presence of errors, but never their absence." • Dijkstra CSC 382: Computer Security

9. White and Black Box Testing White Box Testing • Testing guided by the source code and design documentation. • May be approximated by decompiling or disassembling binary code. Black Box Testing • Testing without access to system code. CSC 382: Computer Security

10. Creating a Test Plan Base Test Plan on Threat Model • Decompose application into components. • Identify entry points for each component. • Use STRIDE classification to determine types of tests to perform for each entry point. • Prioritize testing effort based on risk evaluation for each threat. • Use threat trees and DFDs to guide implementation of specific tests. CSC 382: Computer Security

11. 1. Decompose Application Threat Model provides list of application components. CSC 382: Computer Security

12. 2. Identify Entry Points Threat model provides list of entry points. Classify entry points by risk potential5. CSC 382: Computer Security

13. STRIDE5 Threat Categorization Spoofing ex: Replaying authentication transaction. Tampering ex: Modifying authentication files to add new user. Repudiation ex: Denying that you purchased items you actually did. Information disclosure ex: Obtaining a list of customer credit card numbers. Denial of service ex: Consuming CPU time via hash algorithm weakness. Elevation of privilege ex: Subverting a privileged program to run your cmds. CSC 382: Computer Security

14. 3. STRIDE Test Types Spoofing • Attempt to force application to use no authentication. • Attempt to force application to fall back to less secure authentication protocol. • Can security tokens by replayed? • Can security tokens be brute-forced? Tampering • Is it possible to modify and rehash security token? • Create invalid hashes, MACs, and signatures to verify that they are checked correctly. CSC 382: Computer Security

15. 3. STRIDE Test Types Repudiation • Can you prevent logging or auditing? • Is it possible to create requests that create incorrect data in event log? EOF, newline. Information Disclosure • Attempt to access privileged data. Sniffer, file. • Kill process, then search disk for confidential data left behind due to unclean termination. • Cause application to fail. Do error messages reveal any useful information? CSC 382: Computer Security

16. 3. STRIDE Test Types Denial of Service • Flood application with many requests. • Can malformed input crash the application? • Can external influences (low memory, disk, or other resource limits) cause a failure? Elevation of Privilege • Can you execute data as code? • Can an elevated privilege be forced to invoke a shell? CSC 382: Computer Security

17. 4. Prioritize Testing Effort Use risk evaluation from Threat Model. • Summing attack tree values based on categories from DREAD evaluation system. CSC 382: Computer Security

18. Evaluate Risk with DREAD Damage Potential Extent of damage if vulnerability exploited. Reproducibility How often attempt at exploitation works. Exploitability Amount of effort required to exploit vulnerability. Affected Users. Ratio of installed instances of system that would be affected if exploit became widely available. Discoverability Likelihood that vulnerability will be discovered. CSC 382: Computer Security

19. 5. Test Implementation Attack Trees • Divide threat into testable conditions. Data Flow Diagrams • Show how data moves from entry point to system component impacted by threat. • Indicates security mechanisms to test/bypass. CSC 382: Computer Security

20. Dynamic Analysis: Memory Checking Tools like purify and valgrind check C/C++ code memory access at runtime. • Use of unitialized memory. • Memory access after free()/delete. • Out-of-bounds memory access. • Access to inappropriate areas of stack. • Memory leaks. • Overlapping source and dest pointers in memcpy() and similar functions. CSC 382: Computer Security

21. Fault Injection Fault Injection: Modifying internal or external environment of an application in an attempt to discover flaws. Source-based Fault Injection Modifies application source code to produce faults. Environment Fault Injection Modifies environment (memory, libraries, etc.) to produce faults. Input-based Fault Injection Modifies expected input to produce faults. CSC 382: Computer Security

22. Source-based Fault Injection Inject new code at compile-time or runtime to force faults that would not normally occur. • Cause functions to return unexpected values. • Set internal data to values that trigger faults. CSC 382: Computer Security

23. Environment-based Fault Injection Intercept application calls to OS (requests for memory, disk space, etc.) • Selectively deny resources. • Control system call responses. CSC 382: Computer Security

24. Environment-based Example Attacks Alter application files. • Create files using app’s expected filename. • Delete expected files. • Change permissions. • Replace files with links. • Use filenames longer than MAX_PATH. Block access to shared libraries. • Does application crash? • If it doesn’t crash, what failures occur due to lost functionality? CSC 382: Computer Security

25. Environment-based Example Attacks Limit memory, disk, and network resources. • How does app response to allocation failures? • Does app leak confidential info to swap? • Does no network prevent auditing/logging? Manipulate registry key existence and permissions on MS Windows systems. CSC 382: Computer Security

26. Input-based Fault Injection Send unexpectedly bad input to application. Input should be construed broadly: • Command line arguments. • STDIN and GUI inputs. • Files. • Network sockets and RPC interfaces. • Environment variables. • Registry values. CSC 382: Computer Security

27. Input-based Example Attacks Overly large input. Can you trigger a buffer overflow? Out of range values. Can you create an integer overflow? Unexpected characters. Can you supply any metacharacters that will be used by application in an unsafe manner? Check for default accounts. CSC 382: Computer Security

28. Input-based Example Attacks Modify input semantics. • How does system handle a parameter supplied multiple times? Does it check input once, then use the other version of parameter? • What does application do with unknown parameters/fields? Use non-canonical resource names to check for directory traversal flaws. CSC 382: Computer Security

29. Fuzz Testing Black-box input based testing technique. • Uses random data. • Easily automated. • If application crashes or hangs, it fails. Results of 1995 study9. • 15-43% of utilities from commerical UNIX systems failed. • 9% of Linux utilities failed. • 6% of GNU utilities failed. • 50% of X-Windows utilities failed. CSC 382: Computer Security

30. Browser Fuzz Testing Michael Zalewski’s mangleme CGI14 • Generated streams of malformed HTML. • Most browsers crashed due to: • NULL pointers. • Memory corruption. • Buffer overflows. • Memory exhaustion. • Exception: MSIE CSC 382: Computer Security

31. SPIKE Fault Injection Tool Network protocol fault injection. • Stateless protocols. • Stateful protocols. Scripting language makes it easy to convert sniffed network packets into tests. • Automatic length and checksum calculation. • Script permits user-defined varying of packets: • Permute characters. • Buffer-overflow attempts. CSC 382: Computer Security

32. Web Proxy Fault Injection Point browser at fault injection proxy. Fault injection proxy supports • Cookie viewing and modification. • Form parameters viewing and modification. • Automated SQL injection attempts. • Automated buffer overflow attempts. • Brute-force password guessing. Examples • SPIKE Proxy CSC 382: Computer Security

33. Key Points • The Importance of Testing. • Black and White Box Methods. • Test Plan based on Threat Model: • Entry points. • Test types based on STRIDE. • Prioritization based on DREAD. • Implementation using Attack Trees. • Fault Injection • Source-based Fault Injection • Environment Fault Injection • Input-based Fault Injection • Fuzz Testing CSC 382: Computer Security

34. References • Matt Bishop, Introduction to Computer Security, Addison-Wesley, 2005. • Computer Economics, “Malicious Code Attacks Had $13.2 Billion Economic Impact in 2001,” http://www.computereconomics.com/article.cfm?id=133, 2002. • Wenliang Du and Aditya Mathur, “Vulnerability Testing of Software System using Fault Injection,” COAST TR98-02, 1998. • Mark Graff and Kenneth van Wyk, Secure Coding: Principles & Practices, O’Reilly, 2003. • Greg Hoglund and Gary McGraw, Exploiting Software: How to Break Code, Addison-Wesley, 2004. • Michael Howard and David LeBlanc, Writing Secure Code, 2nd edition, Microsoft Press, 2003. • Koziol, et. al, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Wiley, 2004. • Barton Miller, Lars Fredrickson, and Bryan So, “An Empirical Study of the Reliability of UNIX Utilities,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf, 1990. • Barton Miller, et. al., “Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf, 1995. • John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002. • Jeffrey Voas and Gary McGraw, Software Fault Injection: Inoculating Programs Against Errors, Wiley, 1999. • David Wheeler, Secure Programming for UNIX and Linux HOWTO, http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html, 2003. • James Whittaker and Herbert Thompson, How to Break Software Security: Effective Techniques for Security Testing, Addison-Wesley, 2004. • Michael Zalewski, “Web browsers – a mini-farce,” Bugtraq mailing list posting, http://www.securityfocus.com/archive/1/378632, 2004. CSC 382: Computer Security