Gray, the New Black

1. Gray, the New Black Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011 Gray-Box Web Vulnerability Testing

4. Todo • Define gray-box testing • Why black-box is insufficient • What we built • Examples • Haters club

5. Definitions • Black-box testing • System-level tests • No assumptions about implementation

6. Definitions • White-box testing • Examine implementation • Test components in isolation

7. Definitions • Gray-box testing • System-level tests (like black-box) • Examine implementation (like white-box)

8. The Software Security Game • Objective • Rules vs. Strategy • Playing Field

9. OBJECTIVE: Protect everything OBJECTIVE: Exploit one vulnerability

10. Rules for the Defender • Don’t attack the attacker

11. Rules vs. Strategy Rules • Don’t attack the attacker Strategy • Emulate attacker’s techniques

12. Who wins? • Technology • Expertise

13. Who wins? • Technology • Expertise • Time

14. Who wins? • Technology • Expertise • Time

15. Changing the odds

16. The Defender’s Advantage • Time • Technology • Expertise • InsideAccess

17. Prior Art • 2005: Concolic testing: Sen, University of Illinois • 2008: Microsoft SAGE: Godefroid, MSR • 2008: Test Gen for Web Apps: Shay et al, U. Washington • 2008: Accunetix: Accusensor

18. Access to the Software Allows for ‘Hybrid’ analysis Dynamic Analysis Static Analysis Black-box Approach White-box Approach

19. ‘Hybrid’ Analysis Dynamic Analysis Static Analysis Correlation Engine

21. The ‘Real-Time Hybrid’ Approach Real-Time Analysis Dynamic Analysis Static Analysis Correlation Engine

22. Evolving to Integrated Analysis Application Dynamic Analysis Real-Time Analysis Real-time link • Find More • Fix Faster

23. Find More • Reduce false negatives • Automatic attack surface identification • Understand effects of attacks • Detect new types of vulnerabilities • Privacy violation, Log Forging

24. Attack surface identification /login.jsp /pages/account.jsp /pages/balance.jsp /admin/admin.jsp • File system • Configuration-driven • Programmatic

25. Understand effects of attacks ✗ /admin/admin.jsp ✔ sysadmin$./sh Command Injection

26. Fix Faster • Reduce False Positives • Confirm vulnerabilities • Provide Actionable Details • Stack trace • Line of code • Collapse Duplicate Issues • Tie to root cause

27. Reduce FalsePositives /admin/admin.jsp SQLi? ✔

28. Actionable Details /login.jsp

29. Collapse Duplicate Issues /login.jsp /pages/account.jsp /pages/balance.jsp 1 3 1 2 Cross-Site Scripting

30. JavaBB – Case Study • Open Source Bulletin Board • Additional Vulnerabilities • Finds18 SQL Injection results • Root cause analysis • 18 SQL injection results have 1 root cause

31. Vulnerability Diagnosis Confirmed SQL Injection

32. Actionable Details Parameters Line of Code Stack Trace

33. Yazd – Case Study • Open Source Forum • Additional Attack Surface • Discovers hidden ‘admin’ area • 3 Additional Cross-Site Scripting results • Root cause analysis • Collapses 34 XSS into 24 root-cause vulnerabilities

34. Attack surface identification Hidden ‘admin’ area

35. Collapse Duplicate Issues

36. One More Case Study

37. Future • Automated anti-anti automation

38. The Case Against “Hybrid” • Hard to find attack surface with static analysis • Static/dynamic correlation doesn’t work • Doesn’t help with false positives / false negatives • Nobody will run a software monitor (cheating!)

39. The Case for Gray-Box Testing • Black-box is a losing game • Find more • Attack surface • Vulnerability diagnosis • Fix faster • Root cause analysis • Collapse duplicates

40. Gray, the New Black Brian Chess Founder / Chief Scientist Fortify Software, an HP Company June 22, 2011 Gray-Box Web Vulnerability Testing