Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making

1. Total Cost of Cyber (In)security –Integrating operational security metrics into business decision-making Russell Cameron Thomas Principal, Meritology russell.thomas@meritology.com Mini-Metricon, February 5, 2007 San Francisco, CA

2. To introduce a new approach Influence thought leaders, academic research, and professional practice Stimulate your thinking and inspire hope Build productive bridges between business and IT Show how key concepts of each can be made compatible Take a stand on what will work and what won’t To get your feedback Is this on the right rack? Is it worth pursuing? Does it fit with other approaches to security metrics? To recruit collaborators and advocates Non-purposes Debate the devilish details Debate politics Debate acceptability in “Mainstream” and “Late Adopter” organizations It will take years, of course! Purpose of this Talk Mini-Metricon, San Francisco - Feb 5, 2007

3. The Challenge • Problem: Disconnect between business decision-makers and security specialists regarding value and risk of InfoSec* • “Security directors appear to be politically isolated within their companies” • “They face a challenging search for allies when they need to gain support from upper management for new security initiatives.” • “Companies reported less alignment of security with long-range strategic objectives of the firm.” • “The results suggest that security remains a function that is mired in operations in the eyes of senior executives.” • Result: under-spending, over-spending, misallocation, burden-dumping, denial, and worse… • Fighting the last war • Failures of imagination • Unintended consequences * Conference Board Survey Oct. 2006: “Navigating Risk—The Business Case for Security” Mini-Metricon, San Francisco - Feb 5, 2007

4. The Simplistic Approach is a “Blind Alley”ROSI*, ALE**, and variants n S ^ p(L|ei) Li V = i =incident types i = 1 Loss of Economic Value Expected loss value Probability of loss given incident & exposure • Why a “blind alley”? • Laplace’s Dream: “If only we had more data…” • (see appendix) ROSI* = ∆V / I Security “investment” * “Return on Security Investment” ** “Annualized Loss Expectancy” Example reference: “Calculated Risk - Guide to determining security ROI” - CSO Magazine - December 2002 Mini-Metricon, San Francisco - Feb 5, 2007

5. Two Viewpoints on Economic Risk #1 “Rational Investor” (Capital Asset Pricing, Discounted Cash Flow) • What matters: • D Mean, D variance • Fat part of the curve p(v) random walk value • When: • Quarterly EPS • Earnings volatility • Shorter time periods Normal distributions time change in value #2 “Insurance Actuary” (Ruin Theory, “Iceberg Risk”) • What matters: • Extreme events • Tail of the curve p(v) random walk with “avalanches” value • When: • Credit rating • Solvency • Reserve funds • Longer time periods “Fat Tailed” and skewed distributions time change in value 99% “Ruin” Mini-Metricon, San Francisco - Feb 5, 2007

6. The Core Idea: Three Costs Categories Idealized “Catastrophic” “Self-insurance” “Budgeted” mean 1s 2s 3s 4s 5s 6s 7s Annual Probability 1,000x 1x 10x 100x Total Cost of InfoSec (borrowed from “Value at Risk” concept in Financial Services Risk Management) Mini-Metricon, San Francisco - Feb 5, 2007

7. Budgeted Costs • Q: What is the expected (average) impact of security-related costs on EPS and earnings volatility (+/– budget)? • The rule: costs must already be in the budget* somewhere • Defined to fit the budget and spending approval processes • Results in stable ratio-scale values • Theoretically and practically sound • Applies Activity-based Costing methods • Compatible with accounting practice (GAAP) • Fits discounted cash flow assumptions for multi-year analysis • Good information available (in principle) • Simple Arithmetic ® Tractable and simple to understand • Composable across organization units and systems • “If you are claiming cost reductions, show me whose budget I should cut. If you are claiming revenue increases, show me whose sales quota I should raise.” (Exec VP) * Includes both operating and capital budgets, but excludes cyber insurance or reserves Mini-Metricon, San Francisco - Feb 5, 2007

8. Calculating Budgeted Costs (1) • Aggregate direct costs • Security staff, training, awareness, tools, services, technology, management, threat monitoring, assessments, etc. • Direct cost of predictable and expected loss events and remediation w/ portfolio effects • Use cost driver models for indirect costs • Patch testing, installation, upgrades, etc. • Vendor support costs, 3rd party support • Help desk • New employee screening and hiring process • Indirect costs of predictable and expected loss events with portfolio effects • Negotiate cost allocation rules for bundled and overhead costs • Infrastructure software and hardware costs • Application software • Internal IT development • Legal dept. • Identify costs from unintended consequences and “business prevention” • It’s a judgment call how best to account for these, but they will win credibility! • If possible, use incremental cost analysis, not just total costs • Compare to a base case (e.g. a “barely legal” budget) Mini-Metricon, San Francisco - Feb 5, 2007

9. Calculating Budgeted Costs (2) Modeling indirect costs using cost drivers: e.g.Desktop/Laptop Incidents and Remediation Cost #1: Provisioning Illustrative Cost #2: Help Desk • Benefits: • Simplicity– many fewer budget categories than incident types, scenarios, etc. • Effectiveness – puts attentionon the right levers • Focus– most often, a few cost drivers dominate (80/20 rule). Platform Policy # devices / yr. Awareness Compliance % • Method: • Identify cost drivers using security metrics combined with business operational metrics (e.g. number of new employees, turnover, etc.). • Aggregate and simplify where possible. • Only account for budgeted (forward-looking) costs. Use historical costs as a guide, if available. Mini-Metricon, San Francisco - Feb 5, 2007

10. Calculating Budgeted Costs (3) Modeling indirect costs using cost drivers:e.g.Indirect costs of predictable and expected loss events, with portfolio effects • Benefits: • Simpler calculations • More robust to varying assumptions Abstracted and Aggregated attacks, breaches, incidents Asset: Customer DB Risk Drivers Exposure, given defenses Damage, violations, etc. Cost Drivers • Cost Categories: • Staff (extra headcount) • Customer Service (damage control) • etc. Detection, remediation, etc.. Mini-Metricon, San Francisco - Feb 5, 2007

11. Decision Framework for Budgeted Costs Differential Analysis #3 Lifetime #1 Total Budgeted Costs vs. benchmarks Higher #4 Self-insurance Cost Implications Same Indirect Current Lower Time Direct Current Budget “Barely legal” Budget “ Premium” Budget #2 Budget Optimization Mini-Metricon, San Francisco - Feb 5, 2007

12. Self-Insurance Cost • Q: How much money would you put aside each year into a reserve fund* to avoid a serious decline in credit rating due to low-probability/high-impact losses? • The rule: an actuarially-sound self-insurance premium, given… • Budget-busting loss events • Severe outage, delay in a key new product, loss of major sales contract, etc. • Material to quarterly EPS (> 1% ) • Extreme loss events (short of bankruptcy) that threaten credit rating, etc. • Long-lasting business interruption, executive fraud, earnings restatement, regulatory action, punitive damages, etc. • Interdependencies, correlations (“avalanche effects”), and portfolio effects • Parameters: Maximum risk threshold and time horizon set by top management • “Mark to Model” approach, calibrated by history & “wisdom of the crowds” • A betting man’s judgment: “The race doesn’t always go to the swiftest, but that’s how you bet.” *Analogous to the concept of Economic Capital in financial services Mini-Metricon, San Francisco - Feb 5, 2007

13. Calculating Self-Insurance Cost (1) Annual premium ≈ Pool ÷ (Time Period) Cost distribution curve (if time period is long enough) Estimation Parameters Budget threshold 99th Percentile threshold Time period* 1 2 3 Self-insurance pool (“Value at Risk”) Fund solvency* 5 Shape of the curve 4 Interest rates 6 Magnitude of costs • Modeling: • Distribution curves from parameters • Monte Carlo simulation of self-insurance pool with funding parameters, interest rates, etc. to calculate annual premium • Dominated by largest losses 2 * Policy decisions by top management Mini-Metricon, San Francisco - Feb 5, 2007

14. Calculating Self-Insurance Cost (2) Parameter values change with new information How:A Competitive Marketplace for Models parameter Prediction Markets Bayesian Networks External data bases, benchmarks time Consensus Estimates Statistical analysis of historical loss data Qualitative Reasoning (e.g. Inference to the Best Explanation, Reasoning about Uncertainty, etc.) Simulations Delphi Technique Assessments, Scorecards Mini-Metricon, San Francisco - Feb 5, 2007

15. Ways to Make Self-Insurance Cost “Real” • Link it to real cyber insurance policies • Set up a real self-insurance fund via Finite Risk program* or tradable subordinated debt • Use it as the “glue” for multi-firm “risk sharing” pools • Focused on information sharing and mutual assistance, with incentive instruments • Link to performance management and incentive compensation • Subdivide Self-Insurance Cost into a “Risk Budget” for each org. unit, or • Use it as a “risk adjustment” factor for other performance metrics • Create incentive instruments tied to self-insurance costs or cost drivers for… • Security outsource vendors • Supply chain partners • Channel partners • Customers • Alliance partners • Public disclosure • SEC filings, other regulatory filings • Stakeholder reports • Credit rating agencies • “Cap and Trade” markets *See appendix Mini-Metricon, San Francisco - Feb 5, 2007

16. Catastrophic Costs • Q: How much confidence should we have that the firm can survive InfoSec catastrophes? • The rule: prioritized loss scenarios above a significance threshold that cover the space of possibilities. • Use for business continuity preparation→ agility and robustness • Avoid failures of imagination and “fighting the last war” • Root out unintended consequences • Categorize and prioritize – don’t waste time on precision estimates • Strategic scenario analysis, “war gaming”, etc. • Focus on discovery, “out of the box”, and reframing • Challenge conventional wisdom! • “It’s not what we don’t know that will kill us. It’s what we know that ain’t so”. Mini-Metricon, San Francisco - Feb 5, 2007

17. Risk Management Decisions Gambling Prudence Budgeted Costs Catastrophic Costs Self-insurance Costs Mini-Metricon, San Francisco - Feb 5, 2007

18. A Simple Example – Earthquake Preparation Spend an extra $1,440 per year over 30 yearsfor earthquake loss reduction? *from Monte Carlo simulation • ALE same for both • Simple average says “no” to extra spending Mini-Metricon, San Francisco - Feb 5, 2007

19. Self-insurance Costs (1) Mini-Metricon, San Francisco - Feb 5, 2007

20. Self-insurance Costs (2) Justifies extra spending on maximum preparation Mini-Metricon, San Francisco - Feb 5, 2007

21. Needed: Self-insurance Decision Framework A. Like other insurance Which is more credible? Which leads to better decisions? B. Self-borrowing Mini-Metricon, San Francisco - Feb 5, 2007

22. Summary of the Method • Apply enterprise risk management methods • Break InfoSec costs into three categories: • “Budgeted” • “Self-insurance” • “Catastrophic” • Establish methods, targets, and decision processes for each category • Appropriate to the information and uncertainty involved • The nature of decisions that apply • Link the categories • Use operational metrics plus inference to model costs in each category, as appropriate • Focus energy on continuous organization learning Mini-Metricon, San Francisco - Feb 5, 2007

23. Next Steps • Need more theoretical development and empirical testing • Esp. self-insurance concept, models, and decision rules. • Factoring in impact on revenue, market share, profitability (pricing power), and reputation • Need to standardize “Budgeted Costs” and map to InfoSec assessments and frameworks • Need proofs-of-concept using real companies and real data • Make it work politically • Enterprise Risk Managers = your new best friends • TQM and 6 Sigma Specialists = your allies • CFOs = Status excelsior sponsors • Neutralize or convert opposition (legal department, auditors, etc.) • Lead industries = Financial Services? Supply Chain? other? • Political change role model = Indian Gaming??  • Make it acceptable to the mainstream managers • Q: is it sufficiently promising to continue pursuing? Mini-Metricon, San Francisco - Feb 5, 2007

24. Appendix Russell Cameron Thomas Principal, Meritology russell.thomas@meritology.com Mini-Metricon, February 5, 2007 San Francisco, CA

25. Why Measuring the Value of InfoSec is Hard (1) • Information security (InfoSec) should be seen* as a component of enterprise risk management. • "Risk” is a forward-looking estimate of uncertain loss over a time period (same as the timeframe for return on the assets). • Must cope with all forms of uncertainty and ignorance that apply to actors, assets, threats, vulnerabilities, and learning/adaptation over that timeframe. • InfoSec is a repeating evolutionary game • Between threatening actors (incl. nature) and protecting actors (incl. nature) • Each with an evolving capability set, which may be emergent, nascent, and/or tacit. • The terrain for the security game is threats, vulnerabilities, assets, etc. • Thus, "security" is not a state of the system or the assets. It's how the protecting actors define success in the game over time. • Economics of repeating evolutionary games aren’t well understood yet. They don’t fit existing static equilibrium investment models. They require emergent, dynamic models, e.g. agent-based simulation *From the viewpoint of business value Mini-Metricon, San Francisco - Feb 5, 2007

26. Why Measuring the Value of InfoSec is Hard (2) • InfoSec* is inextricably part of the cyber trust “fur ball”, including • Privacy • Digital Rights • Intellectual Property, brands, reputation, trade secrets • Stakeholder disclosure • … and physical security • Historical loss data, even if copious and available, has limited use • The landscape changes too fast • Low frequency / high impact events matter • Unique events matter • The business value of InfoSec isn’t just loss prevention • Value comes from the ability to support profitable risk taking • e.g. Brakes, condoms • Risk balancing is a reflexive process involving perceptions of risk and reward • Varies dramatically by industry and sector • E.g. a bank vs. a rock quarry *From the viewpoint of business value Mini-Metricon, San Francisco - Feb 5, 2007

27. Blind Alleys and Dirt Roads • “Blind Alleys” look good in concept, but won’t work by themselves • Return on Investment (ROI), Net Present Value (NPV), Payback, etc. • Annualized Loss Expectancy (ALE) • Cyber insurance • Product liability and tort laws (“actual damages”) • “Dirt Roads” work, but just barely • 2x2 or 3x3 matrix categorization of incident types or risks by frequency vs. severity • Assessments using scoring and ranking systems • Balanced scorecards • Strategic scenario analysis and walkthroughs • Are there any “Autobahn” approaches out there? • The null / “realist” hypothesis is “no”, assuming insurmountable problems • “Total Cost of (In)security” might be such an approach Mini-Metricon, San Francisco - Feb 5, 2007

28. Why ALE is Dumb • A Simple Case of Three Loss Event Categories* • Firm Equity = $50 million; Annual Earnings = $5 million; ROE = 10% • Category A: “Common flood” • 50% chance of $10,000 loss = $5,000 ALE • Category B: “100 year flood” • 1.0% chance of $500,000 loss [10% of earnings, 1% of equity] = $5,000 ALE • 26% chance of happening at least once in 30 years • Category C: “10,000 year flood” • 0.01% chance of $50 million loss [100% of equity] = $5,000 ALE • Reason 1: ALE math hides risk drivers • A+B+C = A+A+A = B+B+B = C+C+C = $15,000 ALE [1.5% of earnings] • Conflates simple random walks with random walks with avalanches • “Three independent common risks = three independent catastrophic risks” • Reason 2: Unreliable estimates of low probability events dominate • Lack of data + psychology means estimation errors for the tail are much higher • 50% ® 55% chance for A ® $5,250 ALE • 1.0% ® 2.0% chance for B ® $10,000 ALE (45% chance in 30 years!) • 0.01% ® 0.05% chance for C ® $25,000 ALE • S = $40,250 ALE (2.7 times larger!) *Pareto Distribution, k=1, min = 5,000 Mini-Metricon, San Francisco - Feb 5, 2007

29. Finite Risk Programs The insurance industry offers multi-year self-insurance plans that are commonly called finite risk insurance. The name arises from the fact that the risk transfer is very limited. Therefore, the insured will pay for most (or all) the losses Year 1 time Balance carry-forward Fund established $$$ Operational losses From: “Applying Insurance Modeling Techniques to Quantify OR” Dr Marcelo Cruz, RiskMaths, presented at GARP OR Seminar 18-19 October 2001 London Interest paid Mini-Metricon, San Francisco - Feb 5, 2007

30. Ruin Theory applied to Finite Risk Losses following a certain stochastic process Finite Risk hedging needs Initial Finite Risk capital Percentage of gross income allocated against Finite Risk From: “Applying Insurance Modeling Techniques to Quantify OR” Dr Marcelo Cruz, RiskMaths, presented at GARP OR Seminar 18-19 October 2001 London “ruin” Mini-Metricon, San Francisco - Feb 5, 2007