Security Attribute Evaluation Method: A Cost Benefit Analysis

1. Security Attribute Evaluation Method: A Cost Benefit Analysis Shawn A. Butler Computer Science Department Carnegie Mellon University 27 November 2001

2. M S We always seem to need more security! Don’t we have enough? Hey Boss, we need more security. I think we should get the new Acme 2000 Hacker Abolisher

3. M S What are my alternatives? Trust me, we will be more secure! What is it going to cost? What is the added value?

4. Alternatives? Value? S

5. Problem • Security managers lack structured cost-benefit methods to evaluate and compare alternative security solutions.

6. System Design Available Countermeasures Threats Security Components Prioritized Risks Select Countermeasures Risk Assessment Requirements Policies Outcomes Security Architecture Development Process Develop Security Architecture Security Architecture

7. Threats Prioritized Risks Risk Assessment Outcomes The Multi Attribute Risk Assessment • Determine threats and outcomes • Assess outcome attribute values • Assess weights • Compute threat indices • Sensitivity Analysis

8. Threats Scanning Procedural Violation Browsing Distributed Denial of Service Password Nabbing Personal Abuse Signal Interception : : 29 Threats Outcome Attributes Lost Productivity Lost Revenue Regulatory Penalties Reputation Lives Lost Lawsuits : : Determine Threats and Outcomes

9. Scanning in More Detail .01 = plow  (j=attributesWj  Vj(xj low)) .07 = pexpected  (j=attributesWj  Vj(xj expected)) .00 = phigh  (j=attributesWj  Vj(xj high)) 10,220  (.01 +.07 +.00)  886.57

10. Risk Assessment Results

11. Risks as a Percentage of Threat Index Total

12. But what about the numbers?

13. Sensitivity Analysis is Key!! • How sensitive are the answers to estimation errors? • Does it matter if the estimates are not accurate? • How accurate do they have to be before the decision changes? • When is it important to gather additional information?

14. System Design Available Countermeasures Security Components Select Countermeasures Policies Requirements Security Attribute Evaluation Method (SAEM) • Evaluation Method • Assess security technology benefits • Evaluate security technology benefits • Analyze Costs • Assess coverage • Sensitivity Analysis Prioritized Risks

15. Effectiveness Percentages Security Tech Auth Policy Serv Virtual Priv Net Hardened OS Net Monitors Prxy Firewall Vuln Assess PF Firewall Host IDS Net IDS Auditing Threat Assess Security Technology Benefits

16. Prioritized Technologies

17. Analyze Costs 589  Host IDS  Net IDS  Auditing Threat Index   Auth Policy Server  Smart Cards  Single Sign-on  PKI Cert 0 $20,000 $0 Purchase Cost

18. Assess Coverage

19. Host Intrusion Detection Coverage

20. Auditing Coverage

21. Preliminary Results • Risk Assessment threat indices reflect security manager’s concerns • based on interviews and feedback • Security managers are able to estimate technology benefits • based on experience, organizational skill levels, and threat expectations • Sensitivity Analysis is key to method • based on uncertainty of assumptions