Is finding security holes a good idea?

1. Is finding security holes a good idea? Presented By: Jeff Wheeler CSC 682

2. Outline • Introduction • Vulnerability Lifecycle • Cost of Disclosure • Finding rate to pr • Rate of Vulnerability Discovery • Sources of Error

3. Introduction • Assertions • It is better for vulnerabilities to be found by good guys than bad guys. • Vulnerability finding increases total software quality

4. The life cycle of a vulnerability • Introduction – the vulnerability is first released as part of the software. • Discovery – the vulnerability is found. • Private Exploitation – the vulnerability is exploited by the discoverer or a small group known to him or her. • Disclosure – a description of the vulnerability is published.

5. The life cycle of a vulnerability • Public Exploitation – the vulnerability is exploited by the general community of black hats. • Fix Release – a patch or upgrade is released

6. The life cycle of a vulnerability • These events do not occur strictly in this order. • Ex: software manufacture releases disclosure and fix

7. White Hat Discovery • Discovery, Fix, and Disclosure: Best Case • The vulnerability is discovered by a researcher with no interest in exploiting it. • The researcher notifies the vendor • The vendor releases an advisory and a fix • Public exploitation begins at time of disclosure

8. White Hat Discovery

9. Black Hat Discovery • Discovery, Fix, and Disclosure: Worst Case • The vulnerability is first discovered by someone with an interest in exploiting it. • Black hat community exploitation • Knowledgeable person identifies exploit being used against a system and notifies vendor • The vendor releases an advisory and a fix • Public exploitation begins at time of disclosure

10. Black Hat Discovery

11. WHD versus BHD • WHD eliminates period of Private Exploitation • CBHD – CWHD = Cpriv • Are administrators more likely to patch if they know a vulnerability is being actively exploited? • Total number of vulnerable systems will decline more quickly, minimizing peak exploitation rate

12. Cost-Benefit Analysis of Disclosure • Best Case • White hat discovery, never rediscovered or exploited • Worst Case • Black hat discovery • Cpriv + Cpub

13. Cost-Benefit Analysis of Disclosure

14. From finding rate to pr • Assumption: Vulnerability discovery is a stochastic process. • Overall rate of vulnerability discovery in a particular application is a good estimate for pr • Pr upper bound current percent discovery

15. Determining the Vulnerability Discovery Rate • Assumption: Software undergoes multiple releases • If we assume patches/releases do not introduce new bugs, only fixes, we can assume overall software quality increases with time • How does one determine this rate?

16. Determining the Vulnerability Discovery Rate • ICAT vulnerability metabase • A searchable index of computer vulnerabilities. • Entire database available for public download and analysis • Relevant Information • Rate of discovery over time, Program and version effected • Data Cleansing

20. Sources of Error • Unknown Versions • Bad Version Assignment • Announcement Lag • Severity of Vulnerabilities • Operating System Effects • Packages included with OS, use OS release date instead of package release date • Effort Variability • Different Vulnerability Classes • Data Errors

21. Is it worth disclosing vulnerabilities? • If there is no depletion of vulnerabilities, then disclosing vulnerabilities is always harmful. This implies there is an infinite number of vulnerabilities and pr approaches zero. • If we assume the pool of vulnerabilities is depleting, and all vulnerabilities will eventually be discovered, pr=1, and disclosing vulnerabilities makes sense.

22. Conclusions • This research does not provide sufficient evidence that vulnerability finding and disclosure provides in increase in software security sufficient to offset the effort being invested. • This research does not provide sufficient evidence that vulnerability finding and disclosure is a bad idea.

23. Conclusions • Prefer continuous white hat discovery with no disclosure until exploitation by black hat? • How do we estimate the number of vulnerabilities in an application, both discovered and undiscovered?