Closing the Gap: Analyzing the Limitations of Web Application Vulnerability Scanners

1. Closing the Gap: Analyzing the Limitations ofWeb Application Vulnerability Scanners • David Shelly • Randy Marchany • Joseph Tront • Virginia Polytechnic Institute and State University

2. Purpose • To analyze the limitations of web application scanners by using both a secure and insecure version of a custom-built web application • Identify scanner weaknesses • Improve scanner detection techniques • Reduce false-negatives and false-positives 2

3. Problem • > 60% of attacks seen on Internet aimed at web applications [SANS 2009] • Web application vulnerability scanners are not capable of detecting all of the vulnerabilities and attack vectors that exist • Numerous false-negatives and false-positives • False sense of web application security • Web application flaws remain unpatched

4. Relevant Vulnerabilities • SQL Injection • Cross-Site Scripting (XSS) • Session Management Flaws

5. Web Application Scanners

6. Related Work • Evaluation Applications • WebMaven’s BuggyBank • Foundstone’s Hacme Bank, Hacme Shipping, Hacme Travel • Evaluation Techniques • Scan publicly available web applications • Benchmark system

7. Related Work • Evaluation Applications • WebMaven’s BuggyBank • Foundstone’s Hacme Bank, Hacme Shipping, Hacme Travel • Evaluation Techniques • Scan publicly available web applications • Benchmark system

8. Test Bed Design

9. Client-Side Features • Regular users • Submit, assess, purchase, and categorize products • Password reminder • Register new user • Edit account settings • Administrator users • Add other administrators • Approve/deny uploaded templates and item categorizations • View validated templates

10. Server-Side Features • Database • Users table (10 regular, 1 administrator) • Categories table (12 static, 1 custom) • Categorize table (15 uncategorized, 5 categorized) • Items table (10 available) • Templates • 5 validated item templates • 5 awaiting approval

11. Vulnerabilities Implemented SQL Injection Form input (12) Cookie Variable (8) Session Management Predictable session ID (1) Unprotected variables (9) • Cross-Site Scripting • Reflected (10) • Stored (6) • DOM-based (1)

12. SQL Injection Example

13. XSS Injection Example

14. Session Management Example

15. Session Management Example

16. Session Management Example

17. Methodology Most popular web server technologies: • CentOS 4.4 • MySQL 4.1 • Apache 2.0 • PHP 4.3.9

18. Methodology • Controlled benchmark environment: • Controlled variables • Web server technologies • Independent variables • Number of deliberate vulnerabilities • Dependent variables • Observed false positives and false negatives

19. Methodology • Black-box and White-box Analysis: • Black-box Analysis • Perspective of client-side user • Manipulates user supplied input • White-box Analysis • Source code is available • Verify actual number of vulnerabilities

20. Testing Approach • Phases: • 1. Initialization 3. Classification • 2. Execution 4. Analysis

21. Initialization • 1. Restore the MySQL database to its original state • 2. Delete all client side and server side cookies • 3. Restore the entire web server directory with a clean backup • 4. Restart the web server

22. Execution • 1. Configure the web application vulnerability scanner (primarily default settings) • 2. Start Wireshark packet capturing • 3. Execute the web application vulnerability scan • 4. Stop Wireshark packet capturing and save the trace • 5. Save the contents of the database, cookie files, and scanner results

23. Classification • False Positives: • 1. The scan result is due to an application robustness problem (error page, format exception, etc.) and not a vulnerability • 2. Normal operation of the web application results in the same error/problem • 3. Source code analysis determines scan result is incorrect • 4. The results duplicate a vulnerability that has already been accounted for • Note: Repeated input forms that are displayed recursively on a web page are not considered duplicate vulnerabilities if they are detected multiple times by a web application scanner (e.g. user comment boxes)

24. Analysis • HTTP requests and responses • Client and server cookies • Database copies • Scanner result reports

25. SQL Injection Results – Form Inputs

26. SQL Injection Results – Cookie Variables

27. XSS Injection Results - Reflected

28. XSS Injection Results - Stored

29. XSS Injection Results – DOM-based

30. Session Management Results – Predictable SID

31. Session Management Results – Insecure Cookie Variables

32. False Positives – SQL Injection

33. False Positives – XSS Injection

34. Analysis – SQL Injection • False Negatives • Unable to recognize reflected SQL error in response page • Required fields not completed (ex: register and log-in forms need more than one input) • Cookie variables not tested for injection • Improvements • Flag any error or warning response as suspected vulnerability (information disclosure) • All possible combinations of form inputs should be tested • Add cookie variables to list of parameters to be checked

35. Analysis – SQL Injection • False Positives • Duplicate entries reported • Tested both GET and POST methods • Multiple exploit strings for same vulnerability instance • Blind SQL injection returned the same HTML page • Improvements • Only report the implemented HTTP method (unless both methods are supported) • Verify vulnerability using multiple exploit scenarios, but only report SQL injection vulnerability once • Blind SQL injection tests should use time delay statements

36. Analysis – XSS Injection • False Negatives • Required fields not completed • Did not test all possible user supplied input (DOM variable, cookie variables, and uploadable files) • Stored XSS detected through reflected techniques (relies on SQL error pages) • Improvements • Attempt every combination of form inputs • Test all parameters (even search embedded scripts) • Use unique injection strings and then crawl web application multiple times searching for string

37. Analysis – XSS Injection • False Positives • Duplicate entries reported • Same exploit string with different set of parameters (causes request to appear different) • Improvements • Same as SQL injection • Verify vulnerability using multiple exploit scenarios, but only report one XSS vulnerability per input form

38. Analysis – Session Management • False Negatives • Did not attempt to manipulate cookie to perform session hijacking (only tested for information disclosure) • Most determined cookie not set as HttpOnly • Only tested for session management vulnerabilities using information from initial connection • Improvements • Tamper with all unprotected session variables throughout the application (check if response page different) • Analyze SIDs for strong algorithm

39. Analysis – Other Findings • Possible explanations • Overload of requests • Server not producing proper response pages • Scanners not handling all of server responses • Failures in spidering techniques • Scanners missing parameters and links while crawling • Did not test initial log-in page • Improvements • Monitor current connection speed and throttle number of simultaneous connections • Test all pages and variables within scope of web application

40. Conclusions • Scanners detect simple forms of reflected XSS and SQL injection relatively well • Work still needs to be done to detect non-traditional instances of these vulnerabilities • Most false-positives observed from insecure version of web application • Expected secure version to produce false-positives • Did not test enough of the input parameters • Better to use multiple web application scanners • Combined results better than any one individually

41. Future Work • Expand to include analysis of other web application technologies and server configurations • Web 2.0 technologies (Ajax, Flash, Ruby, Python) • Develop black-box web application scanner • Using proposed techniques

42. Questions • security.vt.edu