Pragmatic Uses for Risk Management Practices

1. Pragmatic Uses for Risk Management Practices • A work in progress !! Presented by Douglas Brown, PMP PMO Head SEC Office of Information Technology 202-551-8176 BrownDo@SEC.gov PMI Silver Spring Chapter 12 April 2006

2. Disclaimer • The Securities and Exchange Commission, as a matter of policy, disclaims responsibility for any private publication or statement by any of its employees. The views expressed herein are those of the author and do not necessarily reflect the views of the Commission or of the author’s colleagues upon the staff of the Commission.

3. Perspectives • SEC IT • Approximately $95 million annually • Went from $40 M to over $100 M 2002-2004 • Seismic culture shift, aftershocks still settling • 103 internal IT employees, 300+ contractors supporting • 4000 agency employees at HQ and 10 regional offices • 1000-3000 contractors and other users • Several thousand “regulated entities” • Millions of individual investors (EDGAR) • 40-60 new projects or new phases annually • 60-100 ongoing projects at any one time

4. Risk Management – by the PMBoK • Risk Identification • Risk Analysis • Risk Response Planning • Risk Monitoring and Control • Generally results in • A Risk Management Plan • That sits on the shelf • But meets a compliance requirement

5. Risk Basics • A good word for OMB-300 • E-300 specifies that project schedules and budgets must be specifically risk-loaded • This is the genesis of SEC’s approach • Differentiate between Risk and Issue • Risk = EVENT or CONDITION that MAY occur and, if it did, would lead to project failing to meet baseline • ISSUE = something that is happening (or, usually, NOT happening) now that will result in failing to meet baseline if not resolved by X date. • ACTION ITEM = something that someone needs to do to carry out their part of the plan. If they do not, it would be a problem but we do not have any reason to believe they will not (or if we do so believe, then we have an ISSUE).

6. SEC OIT Uses for Risk Management • Initial project concept • ROM estimation – FY and TCO • ID most obvious risks • Selection of SDLC/PM style • Pre-Acquisition Review • Account for the 19 OMB risk elements • Derive risk score • Assign risk strategy • Allocate cost and schedule buffers • Control • Evaluate

7. Ranges of Estimates • PMBoK range estimates at various points in SDLC • Budgeting process does not recognize ranges • Conflicting interests: • Pad to avoid failure • Understate to avoid project disapproval • Uncertainty = risk. Recognition is 90% of the battle • Buffers • Program-level unallocated funds

8. Need a number 24-48 months out Estimating cost of 18 servers is easy – but why 18? Software: SLOC meaningless nowadays Function points imply design work largely done Can’t estimate from user requirements – or can we? What about re-use SEC Directions Establishing repository to permit development of parameters Establishing EA maps to functional components to permit identification of re-use opportunities Provide a ROM estimate tool for use in those “not a clue” situations Will be refined over time based on actuals Seeking to work with other agency estimation processes Pre-Select: ROM Estimation

9. ROM estimator Qualitative data entry in green cells - complexity - scope Cost through deployment Buffer TCO ROM

10. Pre-Select: Concept Approval • ROM process identifies buffer (30-150% of base) • PM can request less (usually acquisition-only) • Concept request identifies “most likely reasons why project might fail” – bullet list

11. SDLC and PM Style as Risk Tools • 3 SDLCs • Structured (waterfall) • Iterative (releases of functionality) • Acquisition-only (straight purchases) - assigned at time of pre-select decision • 3 PM Styles • PM-Lite • Custom PM: as needed, based on risk and complexity; Level 1 or Level 2 PM assigned • PM Levels – conform to Acquisition Workforce • Collateral duty • Level 1 – system supporting single SEC office • Level 2 – enterprise or complex functional system • Level 3 – multi-agency (no such project yet)

12. Select: Pre-Acquisition Review • 19 Risk Elements assigned in OMB Circular A-11 • Some overlap but consistency has value • Each area assigned High, Medium or Low for probability and impact • Positive outcomes also assessed, treated as HIGH to protect them • Identify one or more risk statements per area to explain • Explain AVOIDANCE plan for HIGH-HIGH, HIGH-MED risks

13. Risk Assessment L L L L L L L L L L L L L L L L L L L = 4 H M = 2 L = 1 + = 4

14. Assigning risk buffers • Pretty simple: risk score = expected buffer • Review board questions buffers that deviate from the risk score • Last year, tried 1-4 as scale (minimum 19, maximum 76) • For 2006, adjustments: • Accommodating acquisition-only projects BUT experience that most projects WERE under-estimated = revise buffer range to 5% to over 100% • Minimum score 4.75, maximum could be 161 – but highly unlikely to approve project with 4-6 high risk elements • Introduction of pre-acquisition review (more detail available)

15. Future directions in measurement • Specify risk management approaches • HIGH = Avoid • Moderate (<9) = Mitigate, Transfer • Low (0.25) = Accept • Refine buffer calculations as data gained • Narrow the total size of buffer assigned by ROM • Reward PMs for declaring and returning buffer – without encouraging padding to get reward • Reward contractors for early delivery under budget with incentive-based contracts

16. Risk Management in Control Phase • Risk log for regular risks • Project dashboard • Performance light based on schedule and cost buffer consumption • “Management attention” light for PM to declare need for help (risk has become issue) • Customer satisfaction light for customer to sound alarm • Re-evaluation at SDLC phase gates

18. Buffer Consumption 100% 0% 0% 100% % complete

19. SDLC Phases and Associated Reviews Steady State Opns Initiation Planning Analysis Design Solution Test Train/Deploy Retirement Acquisition- only Structured Release 1 Release 2 Iterative Release N Output Contract award High-level reqts Business case, Approval Detailed Business & Technical Reqt’s Technical Design Solution OIT Acceptance Business Value Diamonds are mandatory go-forward milestones. GOLD = Formal review; BLUE = sign-offs

20. Evaluation Phase • Project Close-out Reports • Review issues • Compare to initial risk assessments • Gather actual cost and schedule data • Conduct 90-120 Day Operational Assessments

21. Recap: Uses for Risk Management • Initial project concept • ROM estimation – FY and TCO • ID most obvious risks • Selection of SDLC/PM style • Pre-Acquisition Review • Account for the 19 OMB risk elements • Derive risk score • Assign risk strategy • Allocate cost and schedule buffers • Control • Evaluate

22. ?

23. ?

24. Conclusions • Knowing the enemy and yourself • Those who are ignorant of history are doomed to repeat it • Pride goeth before a fall

25. Pragmatic Uses forRisk Management Practices Douglas M. Brown, Ph.D., PMP PMO Head U.S. Securities and Exchange Commission Office of Information Technology 202-551-8176 BrownDo@SEC.gov PMI Silver Spring Chapter 12 April 2006