Vulnerability in the Real World Lessons from both sides of the fence

1. Vulnerability in the Real WorldLessons from both sides of the fence Ryan Permeh Manager of Product Security McAfee Security Architecture Group Ryan_Permeh@McAfee.com

2. Who I am Why should I listen to this guy? • Several years on the Research side • Years as a reverse engineer and exploit developer • Front lines experience as a vendor and developer • Currently manage a large security vendor’s product security. • 29 shipping projects with millions of lines of code

3. Bugs, Bugs, and more Bugs An accelerating trend • Rates of found vulnerabilities increasing • Severity of bugs increasing • Why? • Research isn’t a black art • Attacker tools improving • More software • Do reasons matter that much? Graph from IBM/ISS X-force Threat analysis 2007

4. Examining the Players What motivates Researchers? • Common Good (For the Internet) • Fame • Money • Demonstrate Technical Excellence • Responsibility • Very Bad Things

5. Examining the Players What motivates Vendors? • Common Good (for our customers) • PR and Brand Image • Money • Demonstrate Technical Excellence • Responsibility

6. Today’s Reality It’s not your Daddy’s Vulnerability Market • An emerging economic market for bugs • Fame is less likely • Vendors that make things hard for researchers may find that researchers prefer the market for their bugs • Even efficient vendors may not get reports first • Vendors need to “fill the gap” Lack of reports != Lack of bugs

7. Researcher Relations Seek the common ground • Money from vendors is not likely • Extortion? • Fame from vendors is more likely • Credit in advisories • Acknowledgment in any pr items • Hire researchers with good track records • Many large software companies hire researchers • Avoid antagonism on either side • Disclosure policies and communication

8. Vendors Thinking Like Researchers

9. Growing a Secure Organization Why secure software is good business • Cost of post release vulnerability is oppressive • Implementation in the SDLC is paid for itself by only a few serious bugs • Damage to brand, to customers, and lawsuits An ounce of prevention is worth a pound of cure. Benjamin Franklin ~35x more expensive to fix a bug post release than in design 35x 25x 15x 5x

10. Selling Security in Your Organization Demonstrate a need, then fill the Need Building Security into your process is a long process • Starts Slow • Keep scope realistic for your resources • Demonstrate Risk • Initial reports • Focus on growth, not fear • Increase Scope • [Daily,Weekly,Monthly,Quarterly] wins add up • Keep it Consistent and meet commitments

11. Calculating ROI for Security There ‘Aint No Such Thing as a Free Lunch • Every activity has a cost • Human Capital • Technical Capital • Cost of Opportunity • Focus on high ROI activities • Plan for the short term and the long term • Some activities have short term ROI, others take longer • To justify increases, you must have the data • Clear improvements justify costs and allow for additional resources

12. Automated Repeatable Track over Time Security Report Card Number of external security bugs Number of internal security bugs Bugs per KLOC Fuzzing Coverage % Automated tools coverage % Input Validation Coverage % Cost per Sev 1 (internal and external) Cost per Developer Trained Cost per Penetration test Cost per … Tracking Security Metrics What to measure and why

13. Hand Auditing Relatively low initial cost Relatively high ongoing costs Poor scaling Variable output levels Deep analysis possible Relatively poor ROI Dependant on quality of auditors Static Code Analysis Relatively high initial cost Relatively low ongoing costs Good scaling Consistent output levels Deep analysis usually not possible Reasonable ROI Dependant on quality of tool How do researchers find bugs? Code Audits

14. How do researchers find bugs? Reverse Engineering • Useful bug finding skill • Usually not necessary if you have code • Good for analysis of 3rd party libraries (check the license) • Can find very deep bugs • Specialized, expensive skill to keep on staff • Audits can take longer • Tools are getting better, but still require skills • Pretty low ROI

15. How do researchers find bugs? Fuzzing • Large scale fault injection • Use a public framework or write your own • Great for covering • File formats • Network servers • Web input testing • Find bugs while you sleep • Not very precise • Variable initial cost • Low ongoing cost • Good ROI

16. How do researchers find bugs? Threat Models • Structural /Architectural Analysis with threat enumeration • Pioneered by Microsoft • Discovers architectural flaws and missing pieces • Can be very formal or less so • Scope can be focused or “big picture” • Utilize your architects, developers and QA • Very low initital and ongoing costs • Very good ROI with quick realization • Some items found may require large effort to fix

17. How Vendors Fix and Reduce Bugs Penetration Testing / Blackbox • Use standard security testing practice • Can be easily integrated into QA cycle • Requires some specialized knowledge • Relatively mature tools • Dependant on quality of testers • Good candidate for outsourcing • Variable initial costs • Relatively low ongoing costs (unless outsourced) • Fair ROI with good short term gains • Pairs very nicely with a threat model

18. How Vendors Fix and Reduce Bugs Training • Training developers in Secure coding techniques • Training QA engineers in security testing techniques • Requires time to train • Requires repetition to lock in skills • Can be sped up with intensives • Microsoft’s “stop and train” • Ultimately it does good in reducing bugs, but its slow • ROI is pretty poor, it takes a long time to recoup • Requires trainers and training materials • Employee turnover affects costs • Screening new employees for security experience helps

19. How Vendors Fix and Reduce Bugs Integration into the SDLC • Strong integration of security at all points of the SDLC • Security Requirements • Threat modeling and Secure Design Principles • Secure Coding Guidelines and Automated analysis • Security Test plans, fuzzing, penetration testing • Vulnerability Management • The end goal, an accumulation of all else • ROI dependant on implementation, gains are cumulative and long term

20. Helping Researchers to Understand Vendors

21. Disclosure Policies Full Disclosure • Instant information • Helps attackers and bleeding edge defenders • Theoretically trades instant pain for a potentially quicker remediation cycle • Can work with vendors or can be 0day • Gives all involved the chance to understand the attack on a deeper level, allowing them to better understand their risks

22. Disclosure Policies Non Disclosure • No public information, ever • May be held by vendor or researcher (or bad guy) • Silent fixes (if any), no notification • Subject to patch diffing • Anti-Sec • Legal battles • Heavy regulation

23. Disclosure Policies The Middle Ground “Responsible” Disclosure • Covers all ground between full and non disclosure • Not incompatible with full or non disclosure • Dependant on defined policy • Researcher waits until patch • May release full technical details post patch • May not release anything • Middle ground that people can work with

24. Disclosure Policies Some Examples OIS – Organization for Internet Safety • Coalition of Security and Software Companies • Complete end to end disclosure policy • Fully documented • McAfee and many other vendors follow this model RFPolicy • Full disclosure policy IETF • Steve Christey and Chris Wysopal • Draft, not RFC NIAC – DHS • Large scale study

25. Disclosure Response What to do when a bug is found • Have a coordinated security policy in place • Make it easy for researchers to contact you • security@yourcompany.com • Consider using PGP or GPG for communications • Have a public web site with contact details and PGP public key • Consider posting your policy • Paypal • Respond as soon as possible • Acknowledge receipt of issue immediately • Determine Risk • Treat Security bugs as important

26. Communication with Researchers • Communication with researchers is important • Recommend at least weekly updates • Updates should include any status or movement in the fix • Keep honesty and integrity as priorities • Use this Coordinate release schedule Always Remember, researchers are helping you, even if it’s for their own reasons. Communication with researchers keeps them happier and less likely to engage in hostilities.

27. Releasing Details Coordination and Release Process • Credit helpful researchers • Try hard not to slip on schedules • Coordinate release times • Realize time zones may cause difficulties • Be prepared for questions • Customers • Internal resources • Media

28. Baby StepsNext steps and beyond

29. Building Bridges Engaging the Research Community • Keep building bridges • Quality external researchers are good for the software development process • Continue to learn from the research community • Security Conferences • Trainings • Microsoft’s Bluehat • Stay vigilant of trends and changes

30. Questions? Ryan Permeh Ryan_Permeh@McAfee.com http://www.mcafee.com