Quantitative Risk Analysis Sanjay Goel University at Albany, SUNY Fall 2004

1. Quantitative Risk Analysis Sanjay Goel University at Albany, SUNY Fall 2004

2. Course Outline > Unit 1: What is a Security Assessment? • Definitions and Nomenclature Unit 2: What kinds of threats exist? • Malicious Threats (Viruses & Worms) and Unintentional Threats Unit 3: What kinds of threats exist? (cont’d) • Malicious Threats (Spoofing, Session Hijacking, Miscellaneous) Unit 4: How to perform security assessment? • Risk Analysis: Qualitative Risk Analysis Unit 5: Remediation of risks? • Risk Analysis: Quantitative Risk Analysis

3. Quantitative Risk AnalysisOutline for this unit Module 1: Quantitative Risk Analysis and ALE Module 2: Risk Aggregation Module 3: Case Study Module 4: Cost Benefit Analysis and Regression Testing Module 5: Modeling Uncertainties

4. Module 1Quantitative Risk Analysis and ALE

5. Quantitative Risk Analysis and ALEOutline • What is Risk Analysis? • What is Quantitative Risk Analysis? • What are the steps involved? • How to determine the Likelihood of Exploitation? • How to determine Risk Exposure? • How to compute Annual Loss Expectancy (ALE)? • Examples • Gym Locker • Hard Drive Failure • Virus Attack

6. Quantitative Risk Analysis and ALERisk Analysis Definition • Risk analysis involves the identification and assessment of the levels of risks calculated from the known values of assets and the levels of threats to, and vulnerabilities of, those assets. • It involves the interaction of the following elements: • Assets • Vulnerabilities • Threats • Impacts • Likelihoods • Controls

7. Quantitative Risk Analysis and ALERisk Analysis Concept Map • Threats exploit system vulnerabilities which expose system assets. • Security controls protect against threats by meeting security requirements established on the basis of asset values. Source: Australian Standard Handbook of Information Security Risk Management – HB231-2000

8. Quantitative Risk Analysis and ALEQuantitative Risk Analysis • Quantitative risk analysis methods are based on statistical data and compute numerical values of risk • By quantifying risk, we can justify the benefits of spending money to implement controls. • It involves three steps • Estimation of individual risks • Aggregation of risks • Identification of controls to mitigate risk

9. Quantitative Risk Analysis and ALERisk Analysis Steps Security risks can be analyzed by the following steps: • Identify and determine the value of assets • Determine vulnerabilities • Estimate likelihood of exploitation • Compute frequency of each attack (with & w/o controls) using statistical data • Compute Annualized Loss Expectancy • Compute exposure of each asset given frequency of attacks • Survey applicable controls and their costs • Perform a cost-benefit analysis • Compare exposure with controls and without controls to determine the optimum control

10. Quantitative Risk Analysis and ALEDetermining Assets and Vulnerabilities • Identification of Assets and Vulnerabilities is the same for both Qualitative and Quantitative Risk Analysis • The differences in both of these is in terms of valuation: • Qualitative Risk Analysis is more subjective and relative • Quantitative Risk Analysis is based on actual numerical costs and impacts.

11. Quantitative Risk Analysis and ALEDetermine Likelihood of Exploitation • Likelihood relates to the stringency of existing controls • i.e. likelihood that someone or something will evade controls • Several approaches to computing probability of an event • classical, frequency and subjective • Probabilities hard to compute using classical methods • Frequency can be computed by tracking failures that result in security breaches or create new vulnerabilities can be identified • e.g. operating systems can track hardware failures, failed login attempts, changes in the sizes of data files, etc. • Difficult to obtain frequency of attacks using statistical data.Why? • Data is difficult to obtain & often inaccurate • If automatic tracking is not feasible, expert judgment is used to determine frequency

12. Quantitative Risk Analysis and ALEApproaches • Delphi Approach • Probability in terms of integers (e.g. 1-10) • Normalized • Probability in between 0 (not possible) and 1 (certain)

13. Quantitative Risk Analysis and ALEDelphi Approach • Subjective probability technique originally devised to deal with public policy decisions • Assumes experts can make informed decisions • Results from several experts analyzed • Estimates are revised until consensus is reached among experts

14. Quantitative Risk Analysis and ALERisk Exposure • Risk is usually measured as $ per annum and is quantified by risk exposure. • ALE (Annual Loss Expectancy, expressed as: $/year) • If an event is associated with a loss • LOSS = RISK IMPACT ($) • The probability of an occurrence is in the range of: • 0 (not possible) and 1 (certain) • Quantifying the effects of a risk by multiplying risk impact by risk probability yields risk exposure. • RISK EXPOSURE = RISK IMPACT x RISK PROBABILITY

15. Quantitative Risk Analysis and ALEIntangible Assets • Incorporating intangible assets within Quantitative Risk Analysis is difficult as it is hard to put a price on things such as trust, reputation, or human life. • However, it is necessary to put an as accurate a value as possible when factoring these assets within risk analysis as they may be even more important than tangible assets.

16. Quantitative Risk Analysis and ALEComputing ALE • Single Loss Expectancy: Loss to an asset if event occurs • Value of the lost asset = Ci • Impact on the Asset (if event occurs) = Pi • SLE = Ci * Pi • Annualized Rate of Occurrence (ARO) characterizes, on an annualized basis, the frequency with which a threat is expected to occur. • Annualized Loss Expectancy (ALE) computes risk using the probability of an event occurring over one year. • Formulation  • ALE = (SLE)(ARO) • Source: Handbook of Information Security Management, Micki Krause and Harold F. Tipton

17. Quantitative Risk Analysis and ALEExample #1: Gym Locker • Scenario: There is a gym locker used by its members to store clothes and other valuables. The lockers cannot be locked, but locks can be purchased. • You need to determine: • Risk exposure for gym members • Controls to reduce risk

18. Quantitative Risk Analysis and ALEExample #1: Gym Locker, cont’d. • Identify assets and determine value • Clothes $50 • Wallet $100 • Glasses $100 • Sports equipment $30 • Driver’s license $20 • Car keys $100 • House keys $60 • Tapes and walkman $40 ____ • Total Loss/week: $500 • Find vulnerability • Theft • Accidental loss • Disclosure of information (e.g. read wallet) • Vandalism

19. Quantitative Risk Analysis and ALEExample #1: Gym Locker, cont’d. • 4 (once every four months) • 3 (once a year) • 2 (once every three years) • 1 (less than once every 3 years) • Estimate likelihood of exploitation • 10 (more than once a day) • 9 (once a day) • 7 (once a week) • 6 (once every two weeks) • 5 (once a month) • For theft: estimated likelihood is 7 • Figure annual loss: • ~$500 worth of loss each week, ~52 weeks in a year • ~$26,000 loss per year

20. Quantitative Risk Analysis and ALEExample #1: Gym Locker, cont’d. • Determine cost of added security • New lock $5 • Replacement for lost key $10 • On average members lose one key twice a month (24 times per year) • Estimate likelihood of exploitation under added security • The new likelihood of theft could be estimated at a 4. • Cost Benefit Analysis • Revised Losses (including cost of controls) = (500 * 4) + (15*24) = 2360 • Net savings = 26000 – 2360 = 23640

21. Quantitative Risk Analysis and ALEExample #2: Hard Drive Failure • The chance of your hard drive failing is once every three years • Probability = 1/3 • Intrinsic Cost • $300 to buy new disk • Hours of effort to reload OS and software • 10 hours • Hours to re-key assignments from last backup • 4 hours • Pay per hour of effort • $10.00 per hour • Total loss (risk impact) • $300 + 10 x (10+4) = $440 • Annual Loss Expectancy (pa = per annum) • (440 x 1/3)$pa = $147 pa

22. Quantitative Risk Analysis and ALEExample #3: Virus Attack • Situation: Virus Attack on same system • You frequently swap files with other people, but have no anti-virus software running. • Assume an attack every 6 months (Probability = 2 per year) • No need to buy a new disk • Rebuild effort (10 + 4) hours • Total loss = $10 x (10 + 4) = $140 • ALE = ($140 x 2) $pa = $280 pa

23. Quantitative Risk Analysis and ALEQuestions 1 and 2 • Why is it important to quantify risk? • Give the definitions for: • Single Loss Expectancy • Annualized Rate of Occurrence • Annual Loss Expectancy

24. Quantitative Risk Analysis and ALEQuestion 3 • For this situation: • Same system as examples 2 and 3

25. Module 2Risk Aggregation

26. Risk AggregationOutline • How do you determine risk posture? • What is this risk aggregation model? • Matrices • Asset/Vulnerability • Vulnerability/Threat • Threat/Control

27. Risk AggregationRisk Posture • Individual risks aggregated = Total risk posture • True comparison of relative risks of different organizations • Mathematical approach for aggregation provided • Methodology standardized • Data needs to be customized to organization • Controls can reduce the cost of exposure • Need to determine optimum controls for organization • Methodology for determining controls shown next slide • Analysis should be undertaken to see the impact of new projects on security

28. Risk AggregationModel • Let: • A be a vector of loss of an asset where al is the lth asset, s.t., 0 < l < L • V be a vector of vulnerabilities where vk is the kth vulnerability, s.t., 0 < k < K • T be a vector of threats where tj is the jth asset, s.t., 0 < j < J • C be the vector of vulnerabilities where ci is the ith control, s.t., 0 < i < I • Also Mα be the matrix that defines the impact of vulnerabilities (breach in security) on assets, where, αkl is the impact of kth vulnerability on the lth asset • Also Mβ be the matrix that defines the impact of threats on the vulnerabilities, where, βjk is the impact of jth threat on kth vulnerability • Also Mγ be the matrix that defines the impact of a controls (breach in security) on the threats, where, γij is the impact of ith control on the jth threat The notation is graphically explained in the next few slides

29. Risk AggregationModel, cont’d. A (Assets) • Data Collection: • Primary Data from corporations that track financial losses due to different attacks • Secondary Data from the reports of financial loss from organizations like CERT, CSI/FBI and AIG • Data specific to a corporation, could perhaps be classified into different groups of companies akl V (Vulnerabilities) L K • Where akl is the Impact of vulnerability k on given asset l. • i.e. fraction of the asset value that will be lost if the vulnerability is exploited

30. Risk AggregationModel, cont’d. V (Vulnerabilities) • Data Collection: • Threat data and frequency of threats is information that is routinely collected in CERT and other such agencies. • Log data and collected data from the organization itself can be another source of information • Data can also be collected via use of automated monitoring tools bjk T (Threats) K J bjk is the probability that threat j will exploit vulnerability k

31. Risk AggregationModel, cont’d. T (Threats) • Data Collection: • Approximate control data can be procured from various industry vendors who have done extensive testing with tools. • Other sources of data can be independent agencies which do analysis on tools. gij C (Controls) J I gij is the fraction by which controls reduce the frequency of a threat exploiting a vulnerability

32. Risk AggregationModel, cont’d. Then losses if no control exist Then losses if controls exist • = sum  = product

33. Risk AggregationOptimization If ζ is the maximum allocated budget for controls the optimization problem can be formulated as:

34. Risk AggregationQuestion 1 • How would you collect data for the following: • Assets and Values • Potential Threats • Exploitable Vulnerabilities • Possible Controls

35. Module 3Case Study

36. Case StudyOutline • What is the case about? • What would fit into the categories of: • Assets • Vulnerabilities • Threats • Controls • Filling in the matrices • Asset/Vulnerability • Vulnerability/Threat • Threat/Control

37. Case StudyExample • Use the information that you have learned in the lecture in the following case study of a government organization. • Remember these key steps for determining ALE • Identify and determine the value of assets • Determine vulnerabilities • Estimate likelihood of exploitation • Compute ALE • Survey applicable controls and their costs • Perform a cost-benefit analysis

38. Case StudyCase An organization delivers service throughout New York State. As part of the planning process to prepare the annual budget, the Commissioner has asked the Information Technology Director to perform a risk analysis to determine the organization’s vulnerability to threats against its information assets, and to determine the appropriate level of expenditures to protect against these vulnerabilities. The organization consists of 4,000 employees working in 200 locations, which are organized into 10 regions. The average rate of pay for the employees is $20/hr. Cost benefit analysis has been done on the IT resource deployment, and the current structure is the most beneficial to the organization, so all security recommendations should be based on the current asset deployment. Each of the 200 locations has approximately 20 employees using an equal number of desktop and laptop computers for their fieldwork. These computers are used to collect information related to the people served by the organization, including personally identifying information. Half of each employee’s time is spent collecting information from the clients using shared laptop computers, and half is spent processing the client information at the field office using desktop computers. Replacement cost for the laptops is $2,500 and for the desktop is $1,500. Each of the 10 regions has a network server, which stores all of the work activities of the employees in that region. Each server will cost $30,000 to replace, plus 80 hours of staff time. Each incident involving a server costs the organization approximately $1,600 in IT staff resources for recovery. Each incident where financial records or personal information is compromised costs the organization $15,000 in lawyers time and settlement payouts. Assume that the total assets of the organization are worth 10 million dollars. The organization has begun charging fees for the public records it collects. This information is sold from the organization website at headquarters, via credit card transactions. All of the regional computers are linked to the headquarters via an internal network, and the headquarters has one connection to the Internet. The headquarters servers query the regional servers to fulfill the transactions. The fees collected are approximately $10,000 per day distributed equally from each region, and the transactions are uniformly spread out over a 24 hour period.

39. Case StudyExample- Assets (Tangible) • Transaction Revenue- amount of profit from transactions • Data- client information • Laptops- shared, used for collecting information • Desktops- shared, used for processing client information • Regional Servers- stores all work activities of employees in region • HQ Server- query regional servers to fulfill transactions

40. Case StudyExample- Asset Valuations (Cost per Day) Transaction Revenue $10,000 per day Data (Liability) $10 million (total assets of organization) Laptops ½ x 200 (locations) x 20 (employees) x $2,500 (laptop cost) = $5,000,000 Desktops ½ x 200 (locations) x 20 (employees) x $1,500 (desktop cost) = $3,000,000 Regional Servers $30,000 (server cost)x 10 (regions) + 80 (hours) x $20 (pay rate) x 10 (regions)+ $10,000 (transaction revenue) = $326,000 HQ Server $10,000 (transaction revenue) + $100,000 (cost of HQ server) + 80 (hours) x $20 (pay rate) x 10 (regions) = $126,000

41. Case StudyExample- Vulnerabilities • Vulnerabilities are weaknesses that can be exploited • Vulnerabilities • Laptop Computers • Desktop Computers • Regional Servers • HQ server • Network Infrastructure • Software • Computers and Servers are vulnerable to network attacks such as viruses/worms, intrusion & hardware failures • Laptops are especially vulnerable to theft

42. Case StudyExample- Threats • Threats are malicious & benign events that can exploit vulnerabilities • Several Threats exist • Hardware Failure • Software Failure • Theft • Denial of Service • Viruses/Worms • Insider Attacks • Intrusion and Theft of Information

43. Case StudyExample- Controls • Intrusion detection and firewall upgrades on HQ Server • mitigate HQ server failure and recovery • Anti-Virus Software • mitigates threat of worms, viruses, DOS attacks, and some intrusions • Firewall upgrades • mitigates threats of DOS attacks and some intrusions, worms and viruses • Redundant HQ Server • reduces loss of transaction revenue • Spare laptop computers at each location • reduces loss of transaction revenue and productivity • Warranties • reduces loss of transaction revenue and cost of procuring replacements • Insurance • offset cost of liability • Physical Controls • reduce probability of theft • Security Policy • can be used to reduce most threats.

44. Case StudyAsset/Vulnerability Matrix • The coefficients of this matrix are usually based on internal data as well as financial loss organizations • For the current example we will assume data for illustration of the concept • Transactions are mostly associated with the regional servers which store the data, the HQ server which takes all requests, and the network infrastructure with which clients access the data. (.30 each) • Laptops, desktops and software is only associated with the remaining 10% (.033 each) • Data that is located on laptops and desktops make up only 10% of total data because they are only used for collecting and processing. • The regional servers contain all other data. • Other assets are associated at 100% with their respective vulnerabilities. (e.g. laptops with laptops, desktops with desktops, etc.)

45. Case StudyAsset/Vulnerability Matrix, cont’d. • Customize matrix to assets & vulnerabilities applicable to case • Compute cost of each asset and put them in the value row • Determine correlation with vulnerability and asset • Compute the sum of product of vulnerability & asset values; add to impact column

46. Case StudyVulnerability/Threat Matrix • The coefficients of this matrix are usually based on data from the literature, e.g., • if rate of failure of hardware is rf (per unit time) • the number of pieces of hardware is n then • the total number of failed components during a time period is rf*n • the fraction of hardware that fails is rf*n/n= rf • For the current example we will assume data for illustration of the concept • Failure rate of laptops is .001 per day (i.e., one in a thousand laptops encounters hardware failure during a day) • Similarly failure rate of a desktop is .0002 (i.e. 2 in ten thousand desktops would encounter hardware failure in a given day. • Hardware failure can cause loss of software, however, our assumption is that all software is replaceable from backups

47. Case StudyVulnerability/Threat Matrix, cont’d. • We assume that the hardware failure will disrupt the network once every one hundred days • There is 0.3 percent chance that software failure can lead to failure of desktops • We assume that there is a .01 chance of a laptop being stolen, .001 for a desktop, and .0002 for servers. • There is a very low chance that network equipment is stolen since it is kept in secure rooms (.0001) • When equipment is stolen some software may have been stolen as well • We assume that denial-of-service is primarily targeted at servers and not individual machines • We assume that the denial-of-service can disable machines as well as cause destruction of software • Insider attacks are primarily meant to exploit data & disable machines • We assume that the servers have less access thus are less vulnerable to insider attacks

48. Case StudyVulnerability/Threat Matrix, cont’d. • Complete matrix based on the specific case • Add values from the Impact column of the previous matrix • Determine association between threat and vulnerability • Compute aggregate exposure values by multiplying impact and the associations

49. Case StudyThreat/Control Matrix • Some of these controls have threats associated with them. However, these are secondary considerations and we will be focusing on primary threats. • We assume that IDS systems will control 30% of the DOS attacks, 30% of Viruses and Worms and 90% of intrusions • In addition, IDS systems do not impact insider attacks • Anti-Virus Software will prevent 90% of Viruses and Worms. • That upgrades to a firewall will greatly control (90% each) of DOS attacks, as well as Viruses and Worms. It will control 30% of intrusions, but not insider attacks. • A redundant HQ server will control 10% of hardware failure (when the original HQ server fails). This is the same percentage for theft and insider attacks. • Also, a redundant HQ server will help with 80% in cases of DOS attacks on the HQ server. • Spare laptops will assist in cases of hardware failure and theft (30% because of volume).

50. Case StudyThreat/Control Matrix, cont’d. • We assume that warranties will help with 70% of both hardware failure and software failure. While it will assist with the cost of new hardware or software, will not reduce employee time. • It is determined that insurance will be able to control 90% of impacts from the threats of theft, DOS attacks, Virus/Worm attacks, Insider Attacks, and Intrusion. • Physical controls (locks, key cards, biometrics, etc.) will control 90% of theft. • Also, it is assumed that a security policy will assist with 20% of all threats since every policy can have procedures which can assist in prevention. • Customize matrix based on the specific case • Add values from the threat importance column of the previous matrix • Determine impact of different controls on different threats • Multiply (1-impact) throughout threat column and multiply to threat importance to get values.