CMMI Based Process Improvement Risk Management Concept Version 4.1 Executive Level

1. CMMI Based Process Improvement Risk Management ConceptVersion 4.1 Executive Level Mike Bloom and Joe Duquette October 2001 Organization: ESC / EN (MITRE)

2. CMMI Based Process Improvement Risk Management “And through all this welter of change and development, your mission remains fixed, determined, inviolable -- it is to win our wars.” General Douglas MacArthur

3. CMMI Based Process Improvement Risk Management Reasons “If you always do what you’ve always done, you’ll continue to get what you always got” and we can no longer afford it • Only 16% of all Information Technology (computer and software) projects complete on time and on budget • 31% are cancelled before completion • The remaining 53% are late and over budget, with the typical cost growth exceeding the original budget by more the 89% • Average overrun of project budgets was 189% • The average schedule overrun for projects that were in difficulty was 222% • Of the IT projects that are completed, the final product contains only 61% of the originally specified features • If no formal systems engineering effort is included, projects run the risk of 50% to 100% development cost overruns • DSB recommendation: Employ developers who demonstrate CMM Level 3 or equivalent. Certification must be less that 2 years old. Eliminate the “escape clause” “Charting the Seas of Technology: The CHAOS Study” The Standish Group, January 1995Report of the Defense Science Board Task Force on Defense Software, Nov 2000 INCOSE Systems Engineering Handbook

4. CMMI Based Process Improvement Risk Management Process Improvement Objectives Establish a Standard Risk Management Process That Will Assist in Achieving Overall Center Objectives ESC Objectives  • Shorter Time to Market • Integrated Command and Control and Combat Support • Harmonize Capabilities, Interoperability, User Needs, Budget, and Technology • Dealing With Uncertainty • Life-Cycle Systems Engineering • Streamline Communications Standard Risk Management Process Objectives  • Address Changes in Major Stakeholders • Address Acquisition and Operational Risks • Consistency with Current AF and DOD Policy • Tool Independent • Program Office Perspective • Value-Add Integral to Everyday Program Management • Accessible by Sponsor via Web Technology • Address Risk to the Enterprise    

5. Plan and Prepare for Risk Management Identify and Analyze Risks Establish a Risk Management Strategy Define Risk Parameters Determine Risk Sources and Categories Identify Risks Risk Responsibility Evaluate, Classify, and Prioritize Risks From Project Planning and Project Monitoring and Control Mitigate Risks Implement Risk Mitigation Plans Develop Risk Mitigation Plans DAR CMMI Based Process Improvement CMMI Risk Management Goals and Practices Only Provides A Generic Framework • Risk Management is a continuous, forward-looking process that is an important part of business and technical management processes. Risk management needs to address issues that could endanger critical objectives. A continuous risk management approach is applied to ensure effective anticipation and mitigation of risks with critical impact across the project life cycle. CMMI Continuous Representation, V1.0, August 2000

6. CMMI Based Process Improvement Risk Management Defined • “Risk is a measure of the inability to achieve system life cycle objectives ...” † such as the following: • Assurance of Program Viability • Provision of Operational Capability • Delivery Within Negotiated Baseline • Assurance of Operational Asset Survival • Assurance of Mission Success • Assurance of Personnel Safety and Performance • Assurance of Integration with Operational Environment • “Risk has two components: • The probability (or likelihood) of failing to achieve particular system life cycle objectives • The consequences of failingto achieve those objectives” † Risk • “Risk is a measure of the inability to achieve system life cycle objectives ..” † such as the following: • Assurance of Program Viability • Provision of Operational Capability • Delivery Within Negotiated Baseline • Assurance of Operational Asset Survival • Assurance of Mission Success • Assurance of Personnel Safety and • Performance • Assurance of Integration with Operational • Environment • “Risk has two components: • The probability (or likelihood) of failing • to achieve particular system life cycle objectives • The consequences of failing to achieve those objectives” † †Adapted from AFMCP 63-101, 9 July 1997

7. DoD 5000 Block Lifecycle FOC IOC A B C Concept & Technology Development System Development & Demonstration Production & Deployment Operations & Support FRP Decision Review OT&E Sustainment Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP and Production) Pre - Systems Acquisition CMMI Based Process Improvement Program Life Cycle and Risk Management

8. Concept Def. Acq. Strat. Mission Integ. Package Develop O&S CMMI Based Process Improvement Objectives and the System Life Cycle System Life Cycle Objectives • Assurance of Program Viability • Provision of Operational Capability • Delivery Within Negotiated Baseline • Assurance of Operational Asset Survival • Assurance of Mission Success • Assurance of Personnel Safety and Performance • Assurance of Integration with Ops. Environment

9. = May be involved CMMI Based Process Improvement Process Life Cycle Context Players in the Life of a System When Do They Play Key:  = Must be involved

10. Process Modeling Concept Definition Mission Area Shortfalls & Acquisition Strategy Planning Opportunities Vision & Goals Funding & Direction ORD Risk Zone Experimentation SRD/TRD Acquisition Strategy User Requirements PPBS Development Development Capstone Risk Definition Spiral Feedback Architecture Zone ASP Risk Budgeting & Tradeoffs Threat Change Risk Risk Zone Zone Zone Requirements RFP Mission Shift Source Selection Operational & Risk Feasibility Preparation Technical Requirement Feasibility Existing Direction Architecture(s) Risk Contract Zone Contract Contract Form Program Funding & Direction Program Award Award Award Office Planning IPT Depot Funding & Contract System DT&E Start Award Complete Disposal Direction Contract Contract Contract Package Development Award Award Award System Life Cycle Form Post Award Product Design Product Integration Working Level Conference & Development IPT Define Post Award Work Packages Back to and IPTs Mission Operations & Package Concept Acquisition Development Fielding Partial Development Integration Support Definition Strategy Decision Zones FOC FOC IOC IOC A A B B C C Full Production & Production & Concept & Concept & System Development System Development Deployment Deployment Operations & Operations & Technology Technology & Demonstration & Demonstration Support Support FRP FRP Development Development Decision Decision Production OT&E OT&E Review Review Test Assets DT&E Test Assets Test Assets Readiness Sustainment Systems Acquisition Pre - Systems (Engineering and Manufacturing Acquisition Development, Demonstration, LRIP and Production) Operations & Support IOC Test Assets IOC Mission Integration IOC Test Assets Test Assets Form Begin Certification and Establish Test OT&E Test & Integration Operations Operate Accreditation Environments Form User Working Group and O&M Planning Group Labs Security Maintain Support Elements Begin Net Worthiness Test Assets Interoperability No Production Back to Sustainment Training Development Decision Deficiency Reports Yes Incident Reports Maint Data Analysis TCTOs , Reprocurement , Modifications, Phase Out & Disposal Infrastructure Finalize Production OT&E Phase Out & Disposal Phase Out & Disposal IOC IOC IOC SLEP Engineering, Mission Shift Readiness Readiness System Fixes Conduct Training, Connectivity, Installation, Deployment CMMI Based Process Improvement Program Life Cycle and Risk Management

11. Operational Risk Management Acquisition Risk Management Concept & Technology Development System Development & Demonstration Production & Deployment Operations & Support CMMI Based Process Improvement Risk Management Space Mission Operations & Support Concept Definition Package Development Acquisition Strategy Integration IOC FOC A B C FRP Decision Review OT&E Sustainment Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP & Production) Pre - Systems Acquisition

12. Step 1 Prepare Step 2 Identify Risks & Hazards Yes Revise Risk Plan Step 3 Assess & Prioritize Risks Is the Risk Plan Working ? New Phase or Key Stakeholder ? No Yes Yes (Continue Monitoring) Key Milestone Approaching ? No (Continue Monitoring) No (Continue Monitoring) Step 7 Monitor Handling Plans Step 4 Decide on Control Options (Continue Monitoring) No “n” Mo Since Last Assessment ? Yes Step 6 Implement Risk Handling Plans Step 5 Establish Handling Plans CMMI Based Process Improvement Risk Management Process

13. Operational Risk Management Acquisition Risk Management Concept & Technology Development System Development & Demonstration Production & Deployment Operations & Support CMMI Based Process Improvement System Life Cycle Application of Risk Process Mission Operations & Support Concept Definition Package Development Acquisition Strategy Integration IOC FOC A B C FRP Decision Review OT&E Sustainment Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP & Production) Pre - Systems Acquisition Applied Continuously Through Life Cycle

14. Draft Risk Management Plan Evolves From These Actions PM Buy-In Stakeholder Buy-IN Mission Priority ID Risks CMMI Based Process Improvement Risk Management Process Step 1 - Prepare Action 1: Obtain Buy-In From Program Manger on Risk Assessment and Management Action 2: Identify & Notify Key Program/ Mission Stakeholders Action 3: Identify and Distribute Key Program/Mission Objectives & Requirements Action 4: Identify, Review, and Distribute Applicable Risk /Hazard Taxonomies Think Risks Know the Mission Form the Team Commit • Risk Management Becomes a Management Priority • Manager Becomes Advocate of Risk Management • Manager Commits Energy and Resource to Effort • Risk Management Becomes a Program Priority • Stakeholders Become Co-Sponsors • Stakeholders Commit to Sufficient Resource • Risk Management Becomes a Mission Priority • Process is Focused on Successful Mission • Stakeholders Become Familiar with Program and Mission • Stakeholders Identify Mission Uncertainties • Risk Manager Makes Various Risk Data and Information Available to All Stakeholders • Each Stakeholder Formulates Individual Concerns/Uncertainties

15. CMMI Based Process Improvement Risk Management Process Step 2 - Identify the Risks and Hazards Action 1: Assemble Stakeholders for Risk Assessment Action 2: Review Program/Mission Objectives, Taxonomies and Risk Assessment Process Action 3: Conduct Risk Identification Action 4: Group Related Risks Action 5: Consolidate Related Risks & Write “If - Then” Risk Statements Write Identify Classify Develop Understanding Establish Team a • Conduct Risk Management Meetings • Initial Meeting Sets Tone and Opens Channels of Communication • Subsequent Meetings Need to be Held as Program Progresses • Understand Mission, How Risk Will be Managed, and Tools Available • Compare Current Mission with Past Missions (Taxonomies) • Make Sure Everyone Understands How Risk Management Will be Done • Identify Program Risks • The Inability to Achieve Program Objectives • When will it Happen • Classify the Risks • Can Use a Predefined Structure (Taxonomy, WBS, CPT etc.) • Can Use a Self-Organized Structure • Consolidate Like Risk and Write Risk Statements • Capture Concise Description to be Acted Upon • Risk Statement = Condition + Consequence

16. CMMI Based Process Improvement Risk Management Process Step 3 - Assess and Prioritize Risks Action 1: Identify & Get Consensus on Impact / Severity for Each Risk Action 2: Identify & Get Consensus on Probability of Occurrence for Each Risk Action 3: Identify Time Window when Risk Could Occur Action 4: Reassess Any Existing Risks in Database Action 5: Prioritize Risks by Impact, Probability & Time Action 6: Identify Handling Bands Coarse Sort Old Risks ? Prioritize Probability ? When ? Impact ? A • Identify Consequences or Level of Impact to the Program If the Risk Occurs • Establish or use Predetermined Impact Categories (e.g. Critical, Serious, Moderate, Minor, Negligible) • Determine the Probability of Occurrence • Establish or Use Predetermined Probability Bands ( e.g. Very Unlikely, Unlikely, Probably, Likely, Very Likely) • For Each Risk Identify the Time Period When the Risk Is Likely to Occur • Establish or Use Predetermined Time Periods ( e.g. Near, Midterm, Far) • Incorporate Existing Identified Risks With Newly Identified Risks • Reassess Existing Risks Following Actions 1,2, and 3. • Fold Existing Risks and Newly Identified Risk Together • Prioritize Risks • Involves Grouping Risks Using Impact, Probability and Timing • Objective Is to Identify Most Serious Program Risks • Identify Risk Handling Bands • Place Risks in to Appropriate Handling Band • Objective Is to Establish Preliminary Resource Constraints

17. High Medium Low Avoid Avoid Transfer Transfer Assume Mitigate Mitigate Monitor Monitor “ Cues ” Action Plans “Triggers” Cont Plans CMMI Based Process Improvement Risk Management Process Step 4 Decide on Handling Options Action 1: Identify Handling Options Within Each Risk Band Action 2: Identify Which Risks will be Assumed or Watched Action 3: Identify Which Risks will be Avoided, Transferred or Mitigated Action 4: Assign Plan OPRs for Avoided, Transferred, or Mitigated Risks (Active) Action 5: Establish or Update Risk Database Capture Responsibility Easy Risks Hard Risks Options • Choose Risk Handling Options • Decide Which Risk Will: • Be Assumed • Be Watched (set “Triggers” or “Cues”) • Avoided • Transferred • Mitigated • Assign Responsibility for Risk Planning • Avoid Risk - Research, Design, Fund etc. • Transfer Risk - To Whom, Acceptance • Mitigate Risk - Strategy, Resources etc. • Establish and Update a Risk Database Ranked Risks a

18. Integrated DRAFT Approved CMMI Based Process Improvement Risk Management Process Step 5 - Establish Handling Plans Action 1: Develop Draft Handling Plans and Associated Resource Requirements Action 2: Program Manager Review and Approval of Handling Plans Action 3: Handling Plan are Funded, Directed, and Integrated with Program Management Fund, Direct, Integrate Review and Approve Develop Plans & Estimates A • Draft the Handling Plans • Avoided, Transferred and Mitigated Risks • Contingency and Risk Status Change Plans • 1-3 Pages, Standard Format, Matches Database • Program Manager Review and Approval • Program Manager Buy-in of the Handling Plan • Formal Process to Insure That Resources Required Are Allocated • Opportunity to Improve the Handling Plan and Provide Team Perspective • Process Is Iterative and May Require a Number of Changes to Proposed Plans • Can Provide an Opportunity to Expose and Adjudicate Different Points of View • Funded, Directed and Integrated with Program Management • Usually Requires Expenditure of Resources (E.G. Cost Estimates And/or Budget Actions) • For a Handling Plan to Have Impact It Must Be Enforceable • Appropriate Changes to Program Directives and Execution Documents and Monitored

19. Risk Handling CMMI Based Process Improvement Risk Management Process Step 6 - Implement Risk Handling Action 1: Finalize Risk Management Plan & Management Infrastructure Action 2: Provide Mechanism to Monitor Triggers, Cues and Handling Plans Action 3: Implement Handling Plans as Authorized, Funded, & Scheduled Work with Exit Criteria Action 4: Provide Reporting on Handling Plan Results & Progress in Meeting Exit Criteria Monitor Progress Implement Monitoring Approach Finalize RMP a • Complete Risk Management Plan (RMP) • RMP Can Be Completed - the Program Now Has a Good Understanding of Program Risk • Risk Management  Program Management • Provide for a Mechanism to Monitor • Handling Plans • Triggers and Cues • Implement the Handling Plans • Implement = Knowledge + Resources + Authority to Act • Communicate, Communicate, Communicate • Handling Status Review …. TAKE ACTION! • Risk Management Database …. UPDATE!

20. Low Handling Band Accept Do Nothing Until Reassessment High Handling Band ? Medium Handling Band ? Establish Handling Bands No No Periodic Reassessment Yes Yes Establish Cues for Increasing Risk Est. Trigger & Contingency Plan Yes “Accept” Risk ? No “Accept” Risk ? Yes No Handling Plan Monitoring Prepare, Approve and Implement Handling Plan No Trigger Monitoring Cue Monitoring No Monitor Cues Yes No Monitor Trigger Monitor Handling Plans Time to Reassess ? Yes Yes Yes Time to Reassess ? Continue Implementation of Contingency Plan Yes STOP STOP Cue Occurs ? No Yes Time to Reassess ? No Trigger Occurs ? Yes Yes No Yes Work The “Issue” No Reassess Individual Risk Immediately Yes Handling Plan Monitoring Implement Contingency Plan Cue Monitoring Trigger Monitoring Done or OBE ? Done or OBE ? Is It Working ? Is It Working ? CMMI Based Process Improvement Risk Management Process Step 7 - Monitor Handling Plans Action 1: Periodically Review Handling Plan Results Action 2: Stop or Modify Handling Plans and Resources, if required Action 3: Retire Risks When Handling Plans are Successfully Completed Action 4: Update Risk Database for Handling Plan Progress & Risk Retirement a Update Database Retire Modify or Stop Review

21. STEP 7 - MONITOR HANDLING PLANS Step 7 Decisions Action 1: Periodically Review Action Plan Results Action 2: Stop or Modify Action Plans and Resources, if required Action 3: Retire Risks When Action Plans are Successfully Completed Action 4: Update Risk Database for Action Plan Progress & Risk Retirement Step 7 Decisions Step 7 Decisions Step 7 Decisions • PROACTIVE LOOK AHEAD • To Review & Identify Risks • To Create New Handling Plans • Return to Step 2 to Insure • all Risk and Hazards Have • Been Identified Communicate Major Program Risk to Senior Decision Maker New Phase or Key Stakeholder ? Key Milestone Approaching ? “n” Mo Since Last Assessment ? Is This Not Working ? New Phase or Key Stakeholder ? Key Milestone Approaching ? “n” Mo Since Last Assessment ? Is This Not Working ? Rebuild RMP & Re-Establish Buy-In of Present Set of Stakeholders Pre DEC ORD Yes Yes Yes Yes JAN NOV Yes Yes Yes Yes OCT FEB Return to Step 1 Return to Step 2 Return to Step 2 Return to Step 1 Revise Risk Plan Ops. Env. Dev . Post O&M Integ . Period ORD Return to Step 1 Return to Step 2 Return to Step 2 Return to Step 1 Revise RMP MAR SEP AUG APR New Stakeholders Are Added or Changed Throughout the Program Life Cycle Establish A Regular Review Cycle & Go Back Through the Process Starting With Step 2 MAY JUL Return to Step 1 and Revise the Risk Management Plan JUN CMMI Based Process Improvement Risk Management Process Step 7 - Monitor Handling Plans There may be other reasons or events that will require a reworking of the process …….. it is imperative that the program team be open to this possibility and be prepared to fix the problem because …… Risk Management  Program Management

22. CMMI Based Process Improvement Risk Management Process “Life is tough, but it’s tougher if you’re stupid” John Wayne as Sergeant John M. Stryker, USMC, in “The Sands of Iwo Jima”

23. Mission Area Planning JV 2020 EFX, ATD, ACTD Capstone Architecture PPBS User Requirements Definition Process Modeling Shortfalls & Opportunities Vision & Goals STAR Budgeting & Tradeoffs Experimentation Operational & Technical Architecture(s) Spiral Feedback EAF Threat Change Program Direction Requirement Feasibility Existing Direction Mission Shift Program Planning IPT CMMI Process Life Cycle Context Program Life Cycle Concept Definition

24. Acquisition Strategy Capstone Architecture Acquisition Strategy Development SRD/TRD Development (1) User Requirements ASP Funding Ops & Tech Architectures RFP Preparation (2) APB Requirements Feasibility PMD Source Selection Form Program Office Contract Award CMMI Process Life Cycle Context Program Life Cycle

25. Package Development Contract Award System Architecture Development Post Award Conference Package Design & Development Product Integration Define Post Award Work Packages and IPTs Back to Development Package Design & Development Product Integration Test Ready? Partial Form Working Level IPT Full Production Readiness Requirements Refinement Package Tradeoff and Decision Package Consensus and Contract Action Spiral “n” Design Spiral “n” Development DT&E Test Articles CMMI Process Life Cycle Context Program Life Cycle

26. Mission Integration Test Articles OT&E Certification and Accreditation Form Test & Integration Working Group Production Decision No Back to Development Yes Finalize Production Readiness OT&E System Fixes Infrastructure Readiness Depot Start Conduct Training, Connectivity, Installation, Deployment CMMI Process Life Cycle Context Program Life Cycle Establish Test Environments

27. Operations & Maintenance IOC Operate and Maintain Begin Operations O&M Planning Form User Group Begin Sustainment Spiral N+1 (e.g. TCTOs, Reprocurement, Modifications, SLEP Engineering, Mission Shift) Phase Out & Disposal CMMI Process Life Cycle Context Program Life Cycle