The overall classification of this briefing is: Unclassified

1. Analytic Requirements and Challenges to Supporting Risk-Reduction Return on Investment as a Decision and Performance Metric for Infrastructure Protection April 2009 The overall classification of this briefing is: Unclassified

2. Risk Management Framework Identify Assets, Systems, and Networks Set Goals and Objectives Implement Programs Measure Effectiveness Assess Risks Prioritize Scenario Development Risk Analysis Decision Analysis Cost-Effective Investments Non Cost-Effective Investments • Identify and classify risk • Finalize objectives • Identify underlying assumptions • Define risk • Calculate effectiveness of capabilities to reduce risk • Develop options to reduce risk • Evaluate tradeoffs between risk, cost and other factors • Translate decisions into actionable plans Feedback Loop

3. Decision-Making Analogy

4. Military Decision Making Process Scenario Development Risk Analysis Decision Analysis

5. Risk Analysis Assess Risks • Identifies and compares relative risks by scenario • Supports Risk Communication • Helps prioritize risk management activities T x V

6. Layers of Data p(a) x p(s|a) x $$ Probability of Attack Probability of Success given Attack Monetized Consequence ANALYTICS • Quantification of Data to provide rigor • Aggregation of Data into meaningful, logical categories • Raw Data ATTRIBUTES/ JUDGMENTS DATA Threat Vulnerability Consequence

7. Consequence Monetized Value for Consequence ANALYTICS • Monetized Impact Values • Aggregation of Data into meaningful, logical categories or taxonomy • Objective/ Empirical Data Loss of Life Economic Impact Psychological Impact Mission Disruption ATTRIBUTES/ JUDGMENTS DATA Challenge: Quantifying Psychological Impact and Mission Disruption

8. Vulnerability Probability of Success given Attack SHIRA ANALYTICS • Quantification of Data to provide rigor • Aggregation of Data into meaningful, logical categories • Objective Data Security Configurations TRAM SHIRA ECIP ATTRIBUTES/ JUDGMENTS Security Configurations TRAM DATA TRAM Infrastructure Survey Tool Enhanced Critical Infrastructure Protection SHIRA Protective Security Advisors

9. Threat p(c) x p(i|c) = p(a) Probability of Capability Probability Intent given Capability Probability of Attack ANALYTICS • Quantification through expert elicitation • Judgment based upon analysis of but not aggregation of data • Intelligence Data ATTRIBUTES/ JUDGMENTS Capability Intent NCTC DHS/IA JITF-CT FBI • Challenges: • Strategic vs. Tactical Intelligence • Distribution of p(a)

10. Capability p(c) Intent p(i|c) Threat: Distribution of p(a) • Options for Distribution: • T1/T2 • Geographic

11. Risk Mitigation* Risk Mitigation is a process of identifying and evaluating potential projects to reduce the Risk profile of the agency. Primarily a cost-benefit analysis effort, comparing the risk reduction benefit of potential projects with the estimated costs. Goal is to select a set of projects that result in the maximum possible risk reduction for the amount invested - greatest Return on Investment (ROI). Risk Mitigation is an on-going iterative process: Initial projects identified through high-level analysis effort Generalized project descriptions ROM Costs Candidate projects are refined and more accurate estimates developed Cost-benefit analysis updated and continually reevaluated as project descriptions mature *Terrorism Risk Assessment and Management

12. Cost Analysis* Produce comparable cost estimates for proposed solutions Initial estimates are relative “national-average” rough costs to enable comparison Not actual jurisdictional costs Next step should always be to produce “real” cost estimates Lifecycle costs Capture true long-term cost of implementation and operation Allow comparison of infrastructure projects versus manpower projects *Terrorism Risk Assessment and Management

13. Decision Analysis Prioritize Net Benefit = (Change in Risk) – (Cost of the Investment) Cost Effectiveness = (Change in Risk)/ (Cost of the Investment) Return on Investment • Identify and evaluate potential projects to reduce risk • Develop cost estimates for investment options • Estimate risk reduction • Correlate different investment options

14. Decision Analysis Investment Portfolios developed by the following method: Calculate Net Benefit and Cost Effectiveness for each Investment Rank Each Investment in Terms of its ROI Apply Budget Constraint to Select Investments and Develop Portfolios Cost Effectiveness Net Benefit Cost Effectiveness Net Benefit Assess baseline risk for each asset Test Bed Assets $200 Million 1 1 1 1 Evaluate the effect of three potential investments on risk at each asset Calculate Net Benefit and Cost Effectiveness for each investment Investments n n n n

15. Decision Analysis The ROI approach can help inform decisions at more granular levels, such as specific sectors, geographic areas, and types of assets. The chart below provides a specific look at the top risk reducing investments included in the study for the Chemical Sector. Cost-Effective Investments Non Cost-Effective Investments Challenge: National or Regional Investment Strategy vs. Asset-Specific

16. Decision Analysis ECIP Index Components Component Weights (wi) Protective Measures Indexes (PMIi) Weighted PMIi (wi x PMIi) Physical Security (PSPMI) 0.215 56.93 12.27 Security Management (SMPMI) 0.242 34.32 8.31 Security Force (SFPMI) 0.194 70.69 13.69 Information Sharing 0.080 49.07 3.92 Protective Measures 0.063 53.85 3.39 Dependencies 0.206 57.69 11.87 Overall Protective Measures Index (PMI*) PMI*= 53.46 PMI is the overall protective measures index going from 0 (less secure) to 100 (more secure), the xi represent specific levels of achievement for the 6 major component PMIs, and the ai are scaling constants (weights) that specify the relative importance of the 6 major component PMIs • Sector/ Subsector weightings provide decision metrics for investment options • Analysis of investment options per asset in a given sector with the decision metrics may allow strategic investment decisions at Sector/ Subsector levels

17. Strategic Investment Themes Elements of Threat Elements of Vulnerability Elements of Consequence Reduced Risk Possible Reduction Possible Reduction Possible Reduction Unique to Terrorism Unique to Natural Hazards Common to Terrorism and Natural Hazards Estimated Capability Reduce Threat Through Information Sharing Countermeasures Identify and Address Vulnerabilities Through Critical Infrastructure Protection Loss of Life Enhance Response Capabilities for Improved Consequence Management Intent to Attack Target Identification Economic Loss Probability of Occurrence Robustness Type and Intensity Psychological Impacts Mission Disruption Applies to All Components Enhance Communications to Improve Effectiveness of All Strategic Investments

18. Investment Strategies Investment Themes and Focus Areas Investment Strategies 14

19. Summary of Challenges Developing a Decision-Making Process Culture Generalizing from specific data – Danger of Induction Communicating Uncertainty

20. Questions?

21. Backups

22. Threat - Capability P(a) elicited by Attack Method

23. Threat - Intent

24. Infrastructure Survey Tool 24 24

25. IST 25 25