1 / 47

Importance of Testing in Computer Security

Explore the significance of testing in computer security, including white box vs black box testing, test plan creation, code auditing, review process, static analysis tools, and various types of fault injection and fuzz testing.

kunklec
Download Presentation

Importance of Testing in Computer Security

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EECS 4980/6980 Code Reviews and Testing EECS 4980/6980: Computer Security

  2. Topics • Why is Testing Important? • White Box vs Black Box Testing • Creating a Test Plan • Code Auditing • The Review Process • Static Analysis Tools • Memory Testing • Fault Injection • Source-based Fault Injection • Environment Fault Injection • Input-based Fault Injection • Fuzz Testing EECS 4980/6980: Computer Security

  3. Why is Testing Important? Navy shoots down Civilian Airliner. • 1988 • US Vicennes shot down Airbus 320. • Mistook airbus 320 for a F-14. • 290 lives lost. • Why: Software bug – “cryptic and misleading output displayed by the tracking software.” EECS 4980/6980: Computer Security

  4. Why is Testing Important? “I Love You” virus cost $8.75 billion.2 • Result of Microsoft decision to automatically execute binary attachments in Outlook. EECS 4980/6980: Computer Security

  5. Why is Testing Important? • Complexity • Continually increasing. • Windows 3.1 (3mloc) to Windows XP (40mloc) • Extensibility • Plugins. • Mobile code. • Connectivity • Network access. • Wireless networking. EECS 4980/6980: Computer Security

  6. Software Complexity • 5-50 bugs per/kloc8 • 5/kloc: rigorous quality assurance testing (QA) • 50/kloc: typical feature testing EECS 4980/6980: Computer Security

  7. Software Connectivity • Internet connectivity nearly ubiquitous. • 802.11 wireless networking. • Bluetooth wireless personal area networking. • Embedded devices often networked. • Can your cellphone get a virus? • Symbian Bluetooth virus released in 2004. • Can your automobile get a virus? • BT discovery reveals embedded BT devices in high-end cars. EECS 4980/6980: Computer Security

  8. Testing Caveat "Testing can establish the presence of errors, but never their absence." • Dijkstra EECS 4980/6980: Computer Security

  9. White and Black Box Testing • White Box Testing • Testing guided by the source code and design documentation. • May be approximated by decompiling or disassembling binary code. • Black Box Testing • Testing without access to system code. EECS 4980/6980: Computer Security

  10. Creating a Test Plan Base Test Plan on Threat Model • Decompose application into components. • Identify entry points for each component. • Use STRIDE classification to determine types of tests to perform for each entry point. • Prioritize testing effort based on risk evaluation for each threat. • Use threat trees and DFDs to guide implementation of specific tests. EECS 4980/6980: Computer Security

  11. 1. Decompose Application • Threat Model provides list of application components. EECS 4980/6980: Computer Security

  12. 2. Identify Entry Points • Threat model provides list of entry points. • Classify entry points by risk potential5. EECS 4980/6980: Computer Security

  13. STRIDE5 Threat Categorization • Spoofing • ex: Replaying authentication transaction. • Tampering • ex: Modifying authentication files to add new user. • Repudiation • ex: Denying that you purchased items you actually did. • Information disclosure • ex: Obtaining a list of customer credit card numbers. • Denial of service • ex: Consuming CPU time via hash algorithm weakness. • Elevation of privilege • ex: Subverting a privileged program to run your cmds. EECS 4980/6980: Computer Security

  14. 3. STRIDE Test Types • Spoofing • Attempt to force application to use no authentication. • Attempt to force application to fall back to less secure authentication protocol. • Can security tokens by replayed? • Can security tokens be brute-forced? • Tampering • Is it possible to modify and rehash security token? • Create invalid hashes, MACs, and signatures to verify that they are checked correctly. EECS 4980/6980: Computer Security

  15. 3. STRIDE Test Types • Repudiation • Can you prevent logging or auditing? • Is it possible to create requests that create incorrect data in event log? EOF, newline. • Information Disclosure • Attempt to access privileged data. Sniffer, file. • Kill process, then search disk for confidential data left behind due to unclean termination. • Cause application to fail. Do error messages reveal any useful information? EECS 4980/6980: Computer Security

  16. 3. STRIDE Test Types • Denial of Service • Flood application with many requests. • Can malformed input crash the application? • Can external influences (low memory, disk, or other resource limits) cause a failure? • Elevation of Privilege • Can you execute data as code? • Can an elevated privilege be forced to invoke a shell? EECS 4980/6980: Computer Security

  17. 4. Prioritize Testing Effort • Use risk evaluation from Threat Model. • Summing attack tree values based on categories from DREAD evaluation system. EECS 4980/6980: Computer Security

  18. Evaluate Risk with DREAD • Damage Potential • Extent of damage if vulnerability exploited. • Reproducibility • How often attempt at exploitation works. • Exploitability • Amount of effort required to exploit vulnerability. • Affected Users. • Ration of installed instances of system that would be affected if exploit became widely available. • Discoverability • Likelihood that vulnerability will be discovered. EECS 4980/6980: Computer Security

  19. 5. Test Implementation • Attack Trees • Divide threat into testable conditions. • Data Flow Diagrams • Show how data moves from entry point to system component impacted by threat. • Indicates security mechanisms to test/bypass. EECS 4980/6980: Computer Security

  20. Code Auditing • Why code reviews? • HP and AT&T claim 20-30X more effective than testing alone. • IBM discovers 82% of defects in reviews before testing. • Code reviews are good for finding • Requirements errors. • Design flaws. EECS 4980/6980: Computer Security

  21. Code Reviews: People • Moderator • Manages meeting; follows up on issues. • Reader • Paraphrases code during meeting. • Not the author. • Recorder • Records bugs discovered. • Author • Provides context for code; answers questions. • Makes corrections after code review. EECS 4980/6980: Computer Security

  22. Code Review: Process • Initiation • Author submits code for review. • Code must compile (w/o warnings), pass unit tests. • Preparation • Reviewers read code and mark concerns. • Meeting • 150-200 lines/hour. • Rework • Author makes corrections based on bugs found in review. • Followup • Moderator checks corrections. • Code sent to testing if corrections good. EECS 4980/6980: Computer Security

  23. Code Review: Checklists • Security reviews should include checklists of common problems, including: • buffer overflows • integer overflows • input validation • checking return values • resource name canonicalization • race conditions EECS 4980/6980: Computer Security

  24. Code Review: Problems • Requires substantial expertise in area of coding and security to be effective. • Human readers are fallible, and will miss mistakes. EECS 4980/6980: Computer Security

  25. Static Analyis • Solution: Let a program analyze your source code for security flaws. • Range of approaches • Standard compiler warnings and type checking. • Lexing source checkers that look for bad names like strcpy() and gets(). • Parsing source code checkers. • Parsing checkers with annotations. • Formal proof based program verification. EECS 4980/6980: Computer Security

  26. Static Analysis Tools • Lexing source code checkers. • flawfinder • ITS4 • RATS • Parsing checkers + annotation. • cqual • splint EECS 4980/6980: Computer Security

  27. login.c int validatePassword(const char *plain_pass) { return !strcmp( cipher_pass, crypt(plain_pass, cipher_pass) ); } int login() { int nAttempts=0;int maxAttempts=3;char password[64];int success=0; do { printf( "\npassword: " ); gets( password ); success = validatePassword( password ); if( success ) break; } while( ++nAttempts < maxAttempts ); return success; } void main(int argc, char *argv[]) { int success = 0; char username[64]; strcpy( username, argv[1] ); success = login(); if( success ) printf( "Login successful, %s!\n", username ); else printf( "Too many failed login attempts.\n" ); return( success ); } EECS 4980/6980: Computer Security

  28. login.c: splint output Splint 3.1.1 --- 15 Jun 2004 login.c: (in function validatePassword) login.c:22:13: Operand of ! is non-boolean (int): !strcmp(cipher_pass, crypt(plain_pass, cipher_pass)) The operand of a boolean operator is not a boolean. Use +ptrnegate to allow ! to be used on pointers. (Use -boolops to inhibit warning) login.c:22:12: Return value type boolean does not match declared type int: !strcmp(cipher_pass, crypt(plain_pass, cipher_pass)) To make bool and int types equivalent, use +boolint. EECS 4980/6980: Computer Security

  29. login.c: splint output login.c: (in function login) login.c:34:9: Use of gets leads to a buffer overflow vulnerability. Use fgets instead: gets Use of function that may lead to buffer overflow. (Use -bufferoverflowhigh to inhibit warning) login.c:34:9: Return value (type char *) ignored: gets(password) Result returned by function call is not used. If this is intended, can cast result to (void) to eliminate message. (Use -retvalother to inhibit warning) login.c:36:13: Test expression for if not boolean, type int: success Test expression type is not boolean or int. (Use -predboolint to inhibit warning) EECS 4980/6980: Computer Security

  30. login.c: splint output login.c:43:6: Function main declared to return void, should return int The function main does not match the expected type. (Use -maintype to inhibit warning) login.c: (in function main) login.c:51:9: Test expression for if not boolean, type int: success login.c:56:11: Return expression from function declared void: (success) Types are incompatible. (Use -type to inhibit warning) EECS 4980/6980: Computer Security

  31. login.c: splint output login.c:19:6: Variable exported but not used outside login: cipher_pass A declaration is exported, but not used outside this module. Declaration can use static qualifier. (Use -exportlocal to inhibit warning) login.c:21:5: Function exported but not used outside login: validatePassword login.c:23:1: Definition of validatePassword login.c:25:5: Function exported but not used outside login: login login.c:41:1: Definition of login Finished checking --- 12 code warnings EECS 4980/6980: Computer Security

  32. splint: Annotations • Annotations encoded as special comments. • /*@annotation@*/ • Types of annotations • Add or suppress warning messages. • NULL pointer checking via /*@notnull@*/ • Bounds checking: • strcpy: /*@requires maxSet(s1) >= maxRead(s2) @*/ • User-defined • /*@tainted@*/ attribute EECS 4980/6980: Computer Security

  33. Dynamic Analysis: Memory Checking • Tools like purify and valgrind check C/C++ code memory access at runtime. • Use of unitialized memory. • Memory access after free()/delete. • Out-of-bounds memory access. • Access to inappropriate areas of stack. • Memory leaks. • Overlapping source and dest pointers in memcpy() and similar functions. EECS 4980/6980: Computer Security

  34. Fault Injection • Fault Injection: Modifying internal or external environment of an application in an attempt to discover flaws. • Source-based Fault Injection • Modifies application source code to produce faults. • Environment Fault Injection • Modifies environment (memory, libraries, etc.) to produce faults. • Input-based Fault Injection • Modifies expected input to produce faults. EECS 4980/6980: Computer Security

  35. Source-based Fault Injection • Inject new code at compile-time or runtime to force faults that would not normally occur. • Cause functions to return unexpected values. • Set internal data to values that trigger faults. EECS 4980/6980: Computer Security

  36. Environment-based Fault Injection • Intercept application calls to OS (requests for memory, disk space, etc.) • Selectively deny resources. • Control system call responses. EECS 4980/6980: Computer Security

  37. Environment-based Example Attacks • Alter application files. • Create files using app’s expected filename. • Delete expected files. • Change permissions. • Replace files with links. • Use filenames longer than MAX_PATH. • Block access to shared libraries. • Does application crash? • If it doesn’t crash, what failures occur due to lost functionality? EECS 4980/6980: Computer Security

  38. Environment-based Example Attacks • Limit memory, disk, and network resources. • How does app response to allocation failures? • Does app leak confidential info to swap? • Does no network prevent auditing/logging? • Manipulate registry key existence and permissions on MS Windows systems. EECS 4980/6980: Computer Security

  39. Input-based Fault Injection • Send unexpectedly bad input to application. • Input should be construed broadly: • Command line arguments. • STDIN and GUI inputs. • Files. • Network sockets and RPC interfaces. • Environment variables. • Registry values. EECS 4980/6980: Computer Security

  40. Input-based Example Attacks • Overly large input. • Can you trigger a buffer overflow? • Out of range values. • Can you create an integer overflow? • Unexpected characters. • Can you supply any metacharacters that will be used by application in an unsafe manner? • Check for default accounts. EECS 4980/6980: Computer Security

  41. Input-based Example Attacks • Modify input semantics. • How does system handle a parameter supplied multiple times? Does it check input once, then use the other version of parameter? • What does application do with unknown parameters/fields? • Use non-canonical resource names to check for directory traversal flaws. EECS 4980/6980: Computer Security

  42. Fuzz Testing • Black-box input based testing technique. • Uses random data. • Easily automated. • If application crashes or hangs, it fails. • Results of 1995 study9. • 15-43% of utilities from commerical UNIX systems failed. • 9% of Linux utilities failed. • 6% of GNU utilities failed. • 50% of X-Windows utilities failed. EECS 4980/6980: Computer Security

  43. Browser Fuzz Testing • Michael Zalewski’s mangleme CGI14 • Generated streams of malformed HTML. • Most browsers crashed due to: • NULL pointers. • Memory corruption. • Buffer overflows. • Memory exhaustion. • Exception: MSIE EECS 4980/6980: Computer Security

  44. SPIKE Fault Injection Tool • Network protocol fault injection. • Stateless protocols. • Stateful protocols. • Scripting language makes it easy to convert sniffed network packets into tests. • Automatic length and checksum calculation. • Script permits user-defined varying of packets: • Permute characters. • Buffer-overflow attempts. EECS 4980/6980: Computer Security

  45. Web Proxy Fault Injection • Point browser at fault injection proxy. • Fault injection proxy supports • Cookie viewing and modification. • Form parameters viewing and modification. • Automated SQL injection attempts. • Automated buffer overflow attempts. • Brute-force password guessing. • Examples • SPIKE Proxy EECS 4980/6980: Computer Security

  46. Key Points • The Importance of Testing. • Black and White Box Methods. • Test Plan based on Threat Model: • Entry points. • Test types based on STRIDE. • Prioritization based on DREAD. • Implementation using Attack Trees. • Source Code Auditing Techniques and Tools • Fault Injection • Source-based Fault Injection • Environment Fault Injection • Input-based Fault Injection • Fuzz Testing EECS 4980/6980: Computer Security

  47. References • Matt Bishop, Introduction to Computer Security, Addison-Wesley, 2005. • Computer Economics, “Malicious Code Attacks Had $13.2 Billion Economic Impact in 2001,” http://www.computereconomics.com/article.cfm?id=133, 2002. • Wenliang Du and Aditya Mathur, “Vulnerability Testing of Software System using Fault Injection,” COAST TR98-02, 1998. • David Evans and David Larochelle, “Improving Security using Extensible Lightweight Static Analysis,” IEEE Softwware, 2002. • Mark Graff and Kenneth van Wyk, Secure Coding: Principles & Practices, O’Reilly, 2003. • Greg Hoglund and Gary McGraw, Exploiting Software: How to Break Code, Addison-Wesley, 2004. • Michael Howard and David LeBlanc, Writing Secure Code, 2nd edition, Microsoft Press, 2003. • Koziol, et. al, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Wiley, 2004. • Barton Miller, Lars Fredrickson, and Bryan So, “An Empirical Study of the Reliability of UNIX Utilities,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf, 1990. • Barton Miller, et. al., “Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf, 1995. • John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002. • Jeffrey Voas and Gary McGraw, Software Fault Injection: Inoculating Programs Against Errors, Wiley, 1999. • David Wheeler, Secure Programming for UNIX and Linux HOWTO, http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html, 2003. • James Whittaker and Herbert Thompson, How to Break Software Security: Effective Techniques for Security Testing, Addison-Wesley, 2004. • Michael Zalewski, “Web browsers – a mini-farce,” Bugtraq mailing list posting, http://www.securityfocus.com/archive/1/378632, 2004. EECS 4980/6980: Computer Security

More Related