Meeting the NDAA 2010 Mandate to establish an Agile IT Acquisition Process

1. Meeting the NDAA 2010 Mandateto establish an Agile IT Acquisition Process John Weiler Technical Director, Interop. Clearinghouse www.ICHnet.org Vice Chair, IT Acquisition Advisory Council www.IT-AAC.org john.weiler@ICHnet.org 703.768.0400

2. Agenda • Understanding the IT Acquisition Challenge • Implementation Challenges • Business Drivers & Benefits • Enablers & Critical Success Factors • Recommendations for Transformation • Discussion www.IT-AAC.org

3. The IT Acquisition ChallengeWave 3 Technologies can’t be acquired using Wave 2 processes… • We are in early stages of Wave 3 information technology • Mainframe and Client-Server waves remain in place Internet - Cloud • Virtualized compute; global network enabled, plug & play • IT Infrastructure decoupled from Applications • COTS & OSS Integration, Software as a Service • Waves represent many co-dependent technologies, matured over time • Adding functional capability has become easier with each new wave Information Driven Capability Client/Server - Decentralized • Butenterprise infrastructure gaps &vulnerabilities have become more critical • PC enabled and network • Software distributed in both server and client computers • Heavy focus on software development Centralized - Mainframe • Central computer center, slow turn around • One size fits all • Limited reuse of application modules Information Technology Evolution 1950 1960 1970 1980 1990 2000 2010 2020 DoD is using Wave 2 acquisition & budget processes; to acquire Wave 3 capability www.IT-AAC.org

4. DoD IT Acquisition Challenge Assuring Acquisition Outcomes PM Challenges: • How do requirements fold into Capabilities and align with IT Services? • Early identification of high requirements and over specification? • Making architecture relevant to PMs. How does architecture align with the acquisition strategy? • To what extent can current technologies and IT services satisfy my capability gaps (service orientation)? • How do I move from requirement to deployment in 360 days without cutting corners? DoD IT Acquisition Challenges: • DoD 5000, DODAF, TRL & JCIDS designed for WS development & not tuned for non-developmental items or fast paced IT market • Analytical Methods inconsistent and often produce unpredictable errors • AQ Risk aversion inhibits tailoring or speedy acquisition • Time, cost of duplicative redundant decision making. • Failure to define realm of the possible results in over specification and costly program lifecycles Acquisition Warriors circa post 2003 Making Sound Investment Decisions in a Complex, Fast Paced Environment. 12/19/2019 www.IT-AAC.org

5. New SOA/Cloud Computing ParadigmS/W Development gives way to COTS/OSS Application Assembly Y e s t e r d a y Design, Code & Test SOA & Agile Acquisition focuses on COTS integration over development • Focus is Software Development • Code everything to spec • Timeframes 12-24 months • Complexity and rate of change manageable (CMM) • Technology base Stable • Driven by data model & structured methods Software Components & Off the Shelf Products T o d a y Architect, Acquire, Integrate • Focus on Component Assembly & Integration • Model, Evaluate, & Acquire • Timeframes are 12-24 weeks! • Reliance on industry standards • Rate of change is high and accelerating • Increased Agility & Adaptability of Enterprise Systems www.IT-AAC.org

6. Based on technology and standards (IDEF, UML) Poor alignment of stakeholder views No cross-agency or cross-application business process refactoring Assume custom S/W development w/ little COTS No consistency enforcement of EA artifacts (inter- and intra-agency) Does not produce actionable or comparable output Typically waterfall – not iterative Produces monolithic apps – not modular No consistent COTS evaluation and acquisition process Inhibits use of commercial best practices & SW artifacts Drives stove-pipe development Assumes 20 year lifecycle, and control of all “components” JCIDS does not have a bottom up Traditional DoD EA and 5000.2 Processes Ineffective in coping with the fast paced IT market Traditional 5000.2 Process DODAF 2.0 www.IT-AAC.org

7. IT Acquisition Reform Way Forward

8. NDAA 2010 Policy Guidance for IT Acquisition • Stream Lines the cost, schedule and performance reporting for major automated information systems (MAIS), by providing guidance to DOD to designate MAISs that also qualify as major defense acquisition programs as one or the other; • Authorizes the department to enter into agreements with private sector organizations to arrange for the temporary assignment of DOD IT professionals to the private sector, or for private sector IT professionals to be assigned to DOD organizations. This capability reduces skill gaps in mission critical occupations by accelerating learning of industry best practices through direct interactions; • Establishes more meaningful metrics for the acquisition of software-intensive programs by amending the reporting requirement in law through the replacement of references to “initial operational capability” and “full operational capability” with the term “full deployment decision” in order to bring terminology more in line with updated acquisition regulations; and • Requires the DOD to undertake business process re-engineering efforts before approving the acquisition of new business systems. DOD would also be required to undertake such business process re engineering analyses for business systems already approved for acquisition and deployment. www.IT-AAC.org

9. Agile Acquisitions CSFs and Enablers • Services Oriented Architecture • Way of thinking about systems as set of modular services: business, data, infrastructure • A business driven, services oriented style • Focused on shared infrastructure services • Partitions capabilities and standardizes interfaces • COTS & Capability-based Acquisitions (CBA) • Approach to structuring enterprise solutions that increases modularity and adaptability • Focus on outcomes and COTS • Drives use of Service Level Agreements • Required by Clinger Cohen Act • Facilitates alignment of business and technology • Consistent with Industry Best Practices www.IT-AAC.org

10. Enablers and Critical Success Factorsfor Agile Acquisition • Enablers • Technologies Exist to Enable SOA • Commercial components encourages by ATL • Standards & Best Practices Exist - Adopt them • Innovators and COTS providers provide best value • Critical Success Factors • Business Driven EA and AQ Approach (OMB) • Independent Architecture Development (DSB) • Revised Solution Development Lifecycle focused on COTS acquisition/integration (AF SAB) • Mechanism for leverages existing COTS and associated implementation best practices (CCA) www.IT-AAC.org

11. Acquisition Assurance Method (AAM) Decision Analytics for the Agile Acquisition Streamlining and Assuring the IT Acquisition Processes through SOA and Cloud Computing Mitre Assessment of ICH Method “... the concept of the Interoperability Clearinghouse is sound and vital. Its developing role as an honest broker of all interoperability technologies, no matter what the source, is especially needed. Such efforts should be supported by any organization that wants to stop putting all of its money into maintaining archaic software and obtuse data formats, and instead start focusing on bottom-line issues of productivity and cost-effective use of information technology.” www.IT-AAC.org

12. Agility requires Clear Touch points in the Federal Acquisition Lifecycle  Performance Measures, Objectives, Outcomes (PRM)  Business Objectives (BRM)  Funding, Partnering Strategies Strategy Understanding the Business  Identify Best Practices, technology Enablers, and Components  Existing Stake Holders, Business Processes, and Workflows  Existing Delivery and Access Channels (Portfolio) Market Rearch Knowing What’s Possible  Must Have Functions, Features, and Info Exchanges  Short and Long-Term Requirements  Assessment of As-is state: Gap analysis Requirements Model the Business Define the Gaps  Define Component Relationships to BRM  Wiring & Activity Diagrams, Component Arch, Data Arch  To-Be architecture ‘blueprints’ Develop the “Blueprints” Architecture  Define/Align Service Components  Component Common Criteria, SLA  Select COTS based on normalized EA vendor submissions. Acquisition Obtain Components  Prototype Solution Architecture  Verify ROI, business fit  Validate Sequencing Plan Integration Assemble the Components Execution  Deploy  Manage  re-Baseline Execute & Deploy Iterative Development Value-Based Releases Artifacts and Activities www.IT-AAC.org

13. AAM Aligns and Validates Business Needs with Technical Solutions Reference Models Associated Metrics BRM Business Drivers & Metrics (BRM/OV) Performance Metrics User/Integrator Best Practices Core Business Mission Objectives Business Processes & Infrastructure Business Driven Top Down Security Profiles Effectiveness/Efficiency AAM Service Component BRM Service Components & Metrics (SRM/SV) Appl Service Components Layer 1 Infrastructure Service Components Layer N SAIL Solution Frameworks Aligns with business needs Common Criteria Vendor Solution Templates Interoperability, Fit, Finish Technical Solution & Metrics (TV, TRM) BRM Application Layer 1 Common Infrastructure Layer M Secure Solutions www.IT-AAC.org

14. Acquisition Assurance Method (AAM) Outcomes: Improved Analysis, Transparency & Timely Decision Making The Result DBSAE will Achieve w/ CAM How DBSAE will make it Work • Reduce IT Delivery Time to the Warfighter • Reduce Non-essential Requirements • Reduce Total Lifecycle Costs • Reduce Time to Market CAM Guidebooks Acquisition Assurance Method Standardized for: • Efficiency • Sustainability • Measurability A Proven Method: • Standardized • Mentored • How-to’s • Traceable • Transparent Streamline the Business System Acquisition Process Decision-Quality Results 12 mo. 24mo. 36 mo. 12/19/2019 www.IT-AAC.org

15. AAM Streamlines Acquisition Supply Chain A Services Oriented Approach www.IT-AAC.org

16. AAM Predictable Outcomes Conflict free Mechanisms for SOA Common Services & Solution Assessment A Standardized Architecture Method for: Aligning DODAF and FEA-PMO structure Capturing Business Process and Information Sharing Driving SOA Enabled IT Acquisition Lifecycle Assuring Services Integration and Contractor Mgt Capability Prioritization w/ SLAs and metrics Assessment Framework for: Vetted Business Processes and Requirements Assessing Vendor Service Components (COTS, GOTS, Open Source) Discerning Technology Feasibility/Risk Assessment. Verify market ability to perform Service Level Agreements Access to Critical Expertise and Knowledge Sources: Architecture and Acquisition Processes that work Healthcare Informatics Domain Expertise Emerging technologies, innovations and open source markets Industry best practices in IT Infrastructure and SOA Stake Holder facilitation and outreach • Biz Value to PMs Mitigate deployment risk. CAM’s structure provide decision quality data earlier Concentrates on Capabilities and their Importance to Mission Formalizes the Prioritization Process Greater Access to Innovative Solutions Innovative approach for assuring successful IT Acquisitions www.IT-AAC.org

17. Predictable ResultsRepeatable, measurable, sustainable success USAF: Streamlined COTS Acquisition Process. Applied to Server Virtualization. Established optimal arch with ROI of 450% & $458 million savings Navy: Assessment of AFLOAT Program – CANES SOA & Security Strategy Eliminated hi-risk Requirements by 23%, $100Ms in potential savings USAF: Procurement of E-FOIA System using AAM Completed AoA, BCA, AQ Selection in just 4 months BTA: Assessment of External DoD Hosting Options using AAM $300 million in potential savings with minimal investment USMC: AoA and BusCase for Cross Domain, Thin Client Solutions Greatly Exceeded Forecasted Saving in both analysis and acquisition GSA: Financial Mgt System consolidation using AAM. Moved FMS from OMB “red” to “green”. Eliminated duplicative investments that saved $200M JFCOM: MNIS Evaluation of Alternatives for Cross Domain Solutions Evaluated 100’s of Options in 90 days, enabling stake holder buy in and source selection. GPO: Developed Acquisition Strategy for Future Digital System Led to successful acquisition and implementation on time, on budget and 80% cheaper than NARA RMS BTA: Apply AAM to complete AoA and BCA for Hosting and SOA Reduced average cycle time and cost of Analysis by 80% “We have put to practice the AF Solution Assessment Process (ASAP) at the Air Force Communications Agency (AFCA) with some well documented success. It was developed with Interoperability Clearinghouse (ICH) and provides a structured and measurable IT assessment process with the agility to provide decision-quality assessments ranging from quick-looks to more in-depth capability-focused technology assessments and lightweight business case analysis.” General Mike Peterson, AF CIO www.IT-AAC.org

18. Transformation RoadmapConsolidated Recommendations from DSB, IAC and AF SAB www.IT-AAC.org 18

19. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Clear Problem Statement, Capability Gaps • RFI Assessment • Realm of the Possible • Measures of Effectiveness • Other data as Price lists • Determine Sponsor and Stake Holder representatives • Codify Business Problem statement • Validate Project Scope, Timeline, Outcomes • Collect and evaluate existing data from RFI responses and other sources • Deliver Project POAM • Establish Stake Holder Agreement and Success Criteria • Establish Measures of Effectiveness Critical Success Factors Entry Criteria • Initial Data collection • Initial identification of Capabilities • Business Needs & Gaps Exit Criteria - Outcomes • Approval of Project Plan • Approval of Business Problem and Outcome • Criteria: Adequacy of Capabilities or Plan for correction 0 months 1 month 2 months 3 months 4 months www.IT-AAC.org 19

20. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Capture Problem Statement w/Sponsor • Establish Performance Measurements • Document Agency Services Baseline • Determine industry capabilities and metrics • Capture Function Capabilities • Determine level of granularity needed • Hold Requirements WGs w/Sponsor's Key Stakeholders • Publish Capability Analysis Report (CAR)(Requirements and their Justification) • Work papers on: • Justification of Requirements, & Capabilities • Problem Statement Validation Critical Success Factors Entry Criteria • Approved Project Plan and POAM Exit Criteria • Approval of the CAR by the Functional Sponsor • Criteria: Adequacy of Capabilities or Plan for correction 0 months 1 month 2 months 3 months 4 months 20 www.IT-AAC.org

21. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Analysis Group; Service Components reference model mapping to capabilities/requirements • Work papers on: • Results of the Market Survey • Standards of Practice • Industry Benchmarking Data • Standardized Vocabulary for describing service components and basis for establishing SLAs (not in scope) • Refine Capabilities into Service Component solution models (per OMB FEA-PMO) • Conduct Market Survey • Establish Service Component & Groupings • Review RFQ for adequacy of detail • If RFI responses lack depth or breadth, ICH will conduct Industry Outreach and Benchmarking • Construct Service Component Analysis Groups • PMO review Critical Success Factors Entry Criteria • Approved CAR (Validated Capabilities) Exit Criteria • Approval of Service Component by the PM • Criteria: Adequate industry metrics or plan for correction 0 months 1 month 2 months 3 months 4 months 21 www.IT-AAC.org

22. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Hold Functional WG w/Sponsor's Key Stakeholders • Develop Prioritization Weighting Scale • Team Normalized weighting of the Service Components • Document each weights rationale • Capability Prioritization Matrix • Work papers on: • Service Component Prioritization Scale • Rationale for each weight given (traceability) Critical Success Factors Entry Criteria • Approved Analysis Groups, Service Components and Standards of Practice Exit Criteria • Approved Capability Prioritization Matrix • Criteria: Functional Sponsor Approval 0 months 1 month 2 months 3 months 4 months 22 www.IT-AAC.org

23. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Evaluate RFI Responses • Establish alternatives for the assessment • Establish Scoring WG team • Develop Scoring Plan • Score Alternatives + + • Perform Sensitivity Analysis on Scoring Results • Analyze results • Review AoA date points • Present Results to Functional Sponsor - May included Functional WG team • Analysis of Alternative (Compare New/Existing Solutions against Prioritized Capability) • Work papers on: • Scoring Plan • Scoring Rationale • Sensitivity analyses performed • Technology Maturity Assessment Critical Success Factors Entry Criteria • Approved Capability Prioritization Matrix Exit Criteria • Approval of Feasibility Assessment Report by DBSAE/ PMO • Criteria: (1) Assessment Team agreement on the scores. (2) Reference material justifying scores 0 months 1 month 2 months 3 months 4 months 23 www.IT-AAC.org

24. AAM Quality Controls Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination 3 Activities Artifacts/Deliverables • Setup Main Cost Model • Determine the quantities and time frame to be Evaluate • "Setup Sub-Models for direct, indirect & migration cost + savings" • Determine Model's elements related to ROI • Determine the models for each alternative • Collect Data industry data and assumptions • Conduct TCO • Review of Economic Analysis Results Present Results to Functional Sponsor - May included Functional WG team • Develop Economic Analysis Report • Economic Analysis Report • Solution Architecture • Documented CCA compliance • Work papers on: • Model Documentation • Documentation of each Alternative • Documentation on costs developed for the Mode; • Documentation of Industry Metric determined Critical Success Factors Exit Criteria • Approval of Economic Analysis Report by Functional Sponsor • Criteria: (1) Functional Sponsor Agreement • (2) Reference material justifying cost models Entry Criteria • Approved Feasibility Assessment, AoA 0 months 1 month 2 months 3 months 4 months DBSAE Assessment www.IT-AAC.org