Risk Analysis

1. Risk Analysis

2. Assessing Risk • The potential consequences of not assessing and managing risks can include the following: • Failure to attain expected benefits from the project, • Inaccurate project cost estimates, • Inaccurate project duration estimates, • Failure to achieve adequate system performance levels, and • Failure to adequately integrate the new system with existing hardware, software, or organizational procedures.

3. Project Risk Factors • Project size • Team size, organizational departments, project duration, programming effort • Project structure • New vs. renovated system, resulting organizational changes, management commitment, user perceptions • Development group • Familiarity with platform, software, development method, application area, development of similar systems • User group • Familiarity with IS development process, application area, use of similar systems

4. Assessing Risk(Cont.) • Risk can be managed on a project by: • Changing the project plan to avoid risky factors. • Assigning project team members to carefully manage the risky aspects. • Setting up monitoring methods to determine whether or not potential risk is, in fact, materializing.

5. Assessing Risk(Cont.) • The four primary factors associated with the amount of technical risk on a given project are: • Project size, • Project structure, • The development group’s experience with the application and technology area, and • The user group’s experience with systems development projects and the application area (see also Kirsch, 2000).

6. Assessing Risk(Cont.) • Four general rules emerged as technical risk assessments: • Larger projects are riskier than smaller projects. • A system in which the requirements are easily obtained and highly structured will be less risky than one in which requirements are messy, ill structured, ill defined, or subject to the judgment of an individual.

7. Assessing Risk (Cont.) • The development of a system employing commonly used or standard technology will be less risky than one employing novel or nonstandard technology. • A project is less risky when the user group is familiar with the systems development process and application area than if unfamiliar.

8. Assessing Risk(Cont.) Effects of degree of project structure, project size, and familiarity with application area on project implementation risk (Source: Based on 7th Applegate, Austin, and McFarlan. 2007; Tech Republic, 2005.)

9. Risks and Their Implications in the Requirements Process • Insufficient user involvement • Creeping user requirements • Ambiguous requirements • Gold-plating • Minimal requirements • Overlooking user classes • Incompletely defined requirements Unacceptable product Overruns and degraded quality Rework and wasted time Unnecessary features Missing features Dissatisfied customers All of the above

10. Controlling Risk • Project Management • Application of knowledge, skills, tools, and techniques to achieve targets within specified budget and time constraints • Scope- Quality • Time • Cost • Risk

11. Kaiser Permanente Botches Its Kidney Transplant Center Project • Read the case study and then discuss the following questions: • Classify and describe the problems Kaiser faced in setting up the transplant center. What was the role of information systems and information management in these problems? • What were the people, organization, and technology factors responsible for those problems? • What steps would you have taken to increase the project’s chances for success? • Were there any ethical problems created by this failed project? Explain your answer.

12. Quantifying Risk • For each risk event we’d like to estimate two factors • Probability • What are the chances the event will occur? • Impact • What is the cost if it does occur? • Together, these allow us to compute our exposure for the risk event

13. Risk Event Exposure Exposure = Probability x Cost • Note that the exposure is never the actual cost incurred. If the risk event occurs, the total impact is absorbed; if it doesn’t occur, there is no impact. • The exposure is a measure of what to expect “on the average.” It is useful primarily for assessing the potential cost of an aggregate of risk events.

14. Risk Exposure: An Example • Suppose the risk event we’re considering is that we miss a delivery deadline • We estimate the probability that this will occur to be 25% • Suppose further that our assessment of what missing the delivery deadline will cost us is $100,000 • The exposure is then = .25 X 100,000 = $25,000 • Will this risk event ever cost us $25,000? No. • the cost will be $0 if we deliver on time • the cost will be $100,000 if we miss the deadline

15. Risk Exposure: An Example (cont’d) • Suppose now that we have identified 4 main risk events for our project, each with these same characteristics: 25% probability and $100,000 cost if the event occurs. • So our exposure on each risk is $25,000. • We add these to get our total exposure on the project which is $100,000.

16. Qualitative Risk Assessment • Sometimes it is very difficult to make confident estimates of probabilities and impacts associated with risk events • In these cases, a qualitative assessment of these two factors may make more sense • Ranking each of the two factors (low-high, low-medium-high, or similar) in a two dimensional table will still provide a basis for prioritizing risks • Prioritization is crucial to planning and creating risk management strategies

17. Qualitative Risk Assessment High Low L H H H Probability L L H L Low High Impact

18. Question:Prioritizing Using Qualitative Assessments How would you prioritize risk events based on assessments done using the chart on the previous slide? HH HL LH LL HH LH HL LL or

19. Some other sources of risk • Scope incorrectly defined • Incomplete or incorrect requirements • Poor estimates • Team turnover • Changing scope and requirements • Poor project management and planning • New technologies (already mentioned) • Risks should be assessed and “managed” even though they are unpredictable

20. Risk Response Strategies • Perform contingency planning, including: • a contingency budget • schedule alternatives, to include some built-in float • complete emergency responses designed to deal with major areas of risk • Develop workarounds designed to avoid or minimize selected risks • Mitigate risks • mitigation involves adding activities/deliverables to a project to offset the possible effect of a potential risk event • mitigation occurs before the risk event materializes • therefore mitigation costs are incurred whether the risk event occurs or not • a kind of insurance policy against selected risk events • Evade risk (Hope for the Best)

21. Risk Avoidance and Mitigation • Risks should be identified early in the project • Stakeholders should be aware of potential risks and their impacts • They should help devise strategies for either avoiding them or minimizing their impact • Requires planning • Formulate contingency plans for mitigation • Track risk event potential and revise plan accordingly

22. Group Exercise #1 • What do you consider to be the risks in the BEC project as you currently understand it?. • Is this a low, medium, or high risk project? • How would you propose dealing with the risks?

23. Software Risk: A Recent Example On the following several slides, a recent highly publicized software failure is described. Read this case study and we will then discuss its implications for our study.

24. Errant Trades Reveal a Risk Few Expected Downloaded and adapted from NY Times Online, August 2, 2012 Errant trades from the Knight Capital Group began hitting the New York Stock Exchange almost as soon as the opening bell rang on Wednesday. The trading firm Knight Capital recently rushed to develop a computer program so it could take advantage of a new Wall Street venue for trading stocks. But the firm ran up against its deadline and failed to fully work out the kinks in its system, according to people briefed on the matter. In its debut Wednesday, the software went awry, swamping the stock market with errant trades and putting Knight’s future in jeopardy. Knight, founded in 1995, is a leading matchmaker for buyers and sellers of stocks, handling 11 percent of all trading in the first half of 2012. Knight lost three-quarters of its market value in two days, in addition to losing $440 million from the errant trades, and was scrambling to find financing or a new owner.

25. Errant Trades Reveal a Risk Few Expected (cont’d) The fiasco, the third stock trading debacle in the last five months, revived calls for bolder changes to a computer-driven market that has been hobbled by its own complexity and speed. Among the proposals that gained momentum were stringent testing of computer trading programs and a transaction tax that could reduce trading. In the industry, there was a widespread recognition that the markets had become more dangerous than even specialists realized. Some S.E.C. officials are pushing new measures that would force firms to fully test coding changes before their public debut, according to a government official who spoke on the condition of anonymity. While the idea has long been discussed at the agency, it gained traction after the Knight debacle. The S.E.C. applied limited safeguards on trading after the “flash crash” of 2010 sent the broader market plummeting in a matter of minutes. But big investors like T. Rowe Price, members of Congress and former regulators said Thursday that the S.E.C. and the industry had been too complacent and needed to do more to understand and control the supercharged market.

26. Errant Trades Reveal a Risk Few Expected (cont’d) Arthur Levitt Jr., a former chairman of the Securities and Exchange Commission, said that recent events “have scared the hell out of investors” and called for the agency to hold hearings. “I believe this latest event was handled better than the flash crash, but the larger question is whether our markets are adequate to deal with the technology that is out there,” Mr. Levitt said. “I don’t think they are.” Regulators have made changes to the markets over the last two decades that have taken it out of the hands of a few New York institutions and allowed dozens of high-frequency trading firms and new trading venues to dominate the stock market. The high-speed firms like Knight, which connect directly to the servers of the exchanges and are capable of executing thousands of trades a second, are responsible for more than half of all activity in American markets. Companies that have benefited from the fragmentation and computerization of the markets have largely managed to fend off tighter controls by pointing to the steady decline in the cost of trading stocks.

27. Errant Trades Reveal a Risk Few Expected (cont’d) But even people who had previously defended the advances in trading technology said on Thursday that too many problems had been overlooked. In Knight’s breakdown on Wednesday, as well as in the botched initial public offerings of Facebook in May and BATS Global Markets in March, the problems were caused by new computer programs that had not been adequately tested. Currently regulators have no protocol for signing off on new software programs like the one Knight rolled out. “When they put these things out in the world they are really being tried for the first time in a real-life test,” said David Leinweber, the head of the Center for Innovative Financial Technology at the Lawrence Berkeley National Laboratory. “For other complex systems we do offline simulation testing.” Mr. Leinweber has suggested to the S.E.C. that it do this work with the help of the supercomputing facilities at his center. The S.E.C. has recently moved in this direction by contracting with a high-speed trading firm that will provide it with more up-to-date market information.

28. Group Exercise • What factors seem to have led to the Knight Capital Group disaster? • Is lack of adequate testing the only culprit here? • Do you think these issues might relate to requirements? • Could good risk analysis and management have mitigated this outcome? Explain your answer.

29. Requirements-Related Risks • During Requirements Elicitation • Scope creep • Schedule pressures • Completeness and correctness • Defining non-functional requirements (performance, usability) • Unstated or implicit requirements • Customer understanding and agreement • Customer-presented “solutions” vs. actual needs

30. Requirements-Related Risks (cont’d) • During Requirements Analysis • Requirements Prioritization • Technically difficult features • New environments (hardware, software, application area) • During Requirements Specification • Gaps in understanding and expectation • Schedule pressure to proceed before specification is complete • Ambiguous terminology • Design embedded in requirements

31. Requirements-Related Risks (cont’d) • During Requirements Validation • Unvalidated requirements • Poor inspection techniques for requirements • During Change Management • Changing requirements • Poor change process • Unimplemented requirements • Scope creep

32. Team Exercise #2 In this activity, you will work with the proposed new online system to automate seminar registration. • Identify several risks inherent in the development of the online seminar registration system project. • Consider the risks you’ve identified and decide which can potentially be avoided or eliminated, and which will require mitigation strategies. • More specifically, what particular avoidance and mitigation strategies would you suggest for the various risks you’ve identified?

33. ERD Activity: Seminar Registration System Recall our work with the proposed new online system to automate seminar registration for a company that offers seminars at multiple sites and on multiple dates. Here are some features of the proposed system that were gathered at an initial one-hour meeting with the customer. Seminar registration is now handled by mail or by phone, based on seminar brochures sent out in the mail. The customer wishes to implement an online (web-based) enrollment system. A potential seminar enrollee should be able to go the new web site, select a specific seminar and then pay for and enroll in it if space is available. Payment would be made by an online credit card transaction. Currently all seminar information (title, instructor, location, date, time, current enrollment, and maximum enrollment allowed) is held in a Excel spreadsheet. A separate Excel spreadsheet contains some information (name, address, phone, email, credit card info) about previous seminar attendees. The seminar manager requested a new daily report showing the current status of enrollment for all seminars. 33

34. Context DFD Attendee Info* seminar_cancellation Email Sys seminar_cancelled Class Admin registration_decline completion_rosters confirmation_of_cancellation request_for_transcript class_rosters Proposed System catalog_search_req payment_info Corporate Financial System registration_req_w_cc credit Seminar Attendee registration_req_w_corp_pmt action cancellation seminar_cancellation confirmation_of_seat daily_report confirmation_of_cc_pmt new_seminar transcript Seminar Mgr certificate search_results Seminar Info* * Customer will convert current data