Unit Outline Quantitative Risk Analysis

1. Unit OutlineQuantitative Risk Analysis Module 1: Quantitative Risk Analysis and ALE Module 2:Case Study  Module 3: Cost Benefit Analysis and Regression Testing Module 4: Modeling Uncertainties Module 5: Summary

2. Module 3Cost Benefit Analysis & Regression Testing

3. Cost Benefit Analysis & Regression TestingLearning Objectives • Students should be able to: • Understand how to use matrices for cost benefit analysis. • Calculate risk leverage. • Comprehend how regression testing is used.

4. Cost Benefit AnalysisMatrix Cost Benefit Analysis • The exposure before controls is equal to the summation of the aggregate values for impact value x threat value. (Vulnerability/Threat Matrix); In this case, the value is equal to: $251,037.60 • The exposure after controls is equal to the sum of all of the multiplied threat importance values. • For example, in the Hardware Failure column, we will take each of the threat importance values and subtract them each from 1. These values should be multiplied together. (Threat/Control Matrix) • This will give us: (1-.10) x (1 - .30) x (1 - .70) x (1 - .20) = 0.15 • This value will be multiplied by the threat importance value: • 0.15 x $8,122.00 = $1,218.30 • (cost with controls of Hardware Failure) • Do this for all threat columns and then summate • all the values. This value is equal to: $15,851.19

5. Cost Benefit Analysis Risk Leverage • Costs are associated with both: • Potential Risk Impact • Reducing Risk Impact • Risk Leverage is the difference in risk exposure divided by the cost of reducing the risk • Let • rf be the risk exposure after imposing controls • ri be the risk exposure prior to imposing controls • c be the cost of controls Leverage l = (ri-rf)/c • This tells you how many times the reduction in risk exposure is greater then the cost of controls.

6. Cost Benefit Analysis Matrix Example • We are using this equation to calculate cost: • Ci = Csi + Cri x t • Where Ci is the total cost of control i. • Csi is the static (one-time) cost of the control. • Cri is the additional cost per day (maintenance, updates, etc.) for the control. • t is equal to time (if calculating for a year, would equal 365). • We are assuming cost of control values for this example: • Intrusion Detection: $21,000 x 11 + $160 x 11 x 365 = $873,400 • Anti-Virus: $1,876 x 4,000 (laptops & desktops) + $1,876 x 11 (number of servers) = $7,524,636 + 11 x $160 x 365 = $8,167,036 • Firewall Upgrades: $10,000 x 211 + $160 x 211 = $2,143,760 • Redundant HQ Server: $100,000 + $160 x 365 = $158,400 • Spare Laptops: $2,500 x 200 = $500,000 • Warranties (3 year): $100 x 4,000 (laptops & desktops) + $1000 x 10 (regional servers) + $1,200 (HQ Server) = $411,200 • Insurance: $5,000,000 (per 365 days) • Physical Controls: $5,000 x 211 + $160 x 211 x 365 = $13,377,400 • Security Policy (creation, implementation, enforcement): $640 x 365 = $233,600

7. Cost Benefit AnalysisExample #4: Unauthorized access • Scenario: A company uses a common carrier to link to a network for certain computing applications. The company has identified the risks of unauthorized access to data and computing facilities through the network. These risks can be eliminated by replacement of remote network access with the requirement to access the system only from a machine operated on the company premises. The machine is not owned; a new one would have to be acquired.

8. Cost Benefit Analysis Matrix Example • Leverage l = (ri-rf)/c • ri = $251,037.60 x 365 = $91,628,724 • rf = $15,851.19 x 365 = $5,785,684.35 • C = $30,864,796 • $251,037 – $15,851.19 / $30,864,796 = .008 • $91,628,724 - $5,785,684.35 / $30,864,796 = 2.78 • The reduction in risk exposure is almost 3x greater than the cost of controls

9. Cost Benefit AnalysisExample #4: Unauthorized Access Cost/Benefit Analysis for Replacing Network Access

10. Cost Benefit AnalysisExample #4: Unauthorized Access

11. Regression TestingExample #5: Graphical Cost Benefit Analysis • Scenario: This is a case where use of regression testing is being considered after making an upgrade to fix a security flaw. We want to determine if regression testing is economical in this scenario. • Regression Testing means applying tests to verify that all remaining functions are unaffected by the change. • Lets refer to the diagram on the following slide, to compare the risk impact of doing regression testing with not doing it. • Upper part of the diagram • the risk of conducting regression testing • Lower part of the diagram • shows the risks of not doing regression testing

12. Regression TestingExample #5: Cost Savings • In the two cases, one of three things can happen if regression is done: • We find a critical fault • We miss finding the critical fault • There are no critical faults to be found. • For each possibility • Calculate the probability of an unwanted outcome, P(UO). • Associate a loss with that unwanted outcome, L(UO).

13. Risk Exposure L(UO) = $0.5M P(UO) = 0.75 $0.375M Find critical fault L(UO) = $30M P(UO) = 0.05 $1.500M $1.975M Don’t find critical fault yes L(UO) = $0.5M P(UO) = 0.20 $0.100M No critical fault Do regression testing? L(UO) = $0.5M P(UO) = 0.05 $0.125M Find critical fault no L(UO) = $30M P(UO) = 0.75 $16.500M $16.725M Don’t find critical fault L(UO) = $0.5M P(UO) = 0.20 $0.100M No critical fault Regression TestingExample #5: Calculation In our example, if we do regression testing and miss a critical fault in the system (a probability of 0.05), the loss could be $30 million. Multiplying the two, we find the risk exposure for that strategy to be $1.5 million. As the calculations in the figure prove, it is much safer to do regression testing than to skip it. Combined Risk Exposure

14. Cost Benefit Analysis & Regression TestingAssignment • Do a cost benefit analysis based on the matrix that you have created for your own organization.

15. Cost Benefit Analysis & Regression TestingSummary • Cost Benefit Analysis is useful in determining whether the costs of controls is actually beneficial in terms of actual return or savings than the losses incurred by the risks they are meant to mitigate. • Cost Benefit Analysis LEVERAGE = (RISK EXPOSUREbefore reduction – RISK EXPOSUREafter reduction) ________________________________________________ COST OF REDUCTION • Regression Testing • Used for comparing risk impact