Transformation Roadmap for Achieving Sustainable IT Acquisition Reform

1. IT Acquisition Advisory Council A non-profit public/private do-tank Assuring the business value of IT at the Speed of Need Transformation Roadmap for Achieving Sustainable IT Acquisition Reform Assuring Acquisition Agility for COTS and Enterprise Resource Planning John Weiler, Managing Director Kevin Carroll, former Army PEO EIS Dr. Marv Langston, former DoD CIO Gen Ted Bowlds, former AF ESC CMDR Relationship Mgr: Helmut.Mertins@IT-AAC.org www.IT-AAC.org 703 768 0400

2. IT-AAC Public/Private Partnershipassuring the business value of IT “A knowledgeable & elastic IT Community of Practice, working in the public interests to usher in agile methods, real world expertise and innovations needed to transform Federal IT and assure mission outcomes” Honorable Mike Wynne, 21st AFSEC, Co-Founder and Chairman Emeritus, IT-AAC “Together, these steps will help to catalyze a fundamental reform of Federal IT, which is essential to improving the effectiveness and efficiency of the Federal Government” White House, OMB Director

3. Think Tank Purpose Just in Time Expertise - (vs butts in seats) that taps a virtual pool of seasoned experts from non-profits, SDO, academia, NGOs, independent consultants and nontraditional advisors. Partner capabilities are best in class of; capability analysis, innovation research, solution architectures, tech assessment, business case analysis, performance metrics, BPR. Standardized Agile Acquisition Processes – AF, Navy & BTA deemed AAM asmeasurable, repeatable and sustainable. Derived from commercial best practices maintained as an Open Source Offering (on GSA Schedule) Innovation Research – decision analytics templates that define the realm of the possible needed to avoid over specification. Reduces time/cost of TRLs as applied to Open Source and COTS. Risk mitigation via evidenced based research. Acquisition Ecosystem Empowerment – Workshops,Training and Mentoring services using IT-AAC’s Grey Beards. To provide the Decision Makers and Stake Holder with an alternative set of Agile Methods, Decision Analytics and IT Expertise needed to assure rapid delivery of Commercial IT Solutions “You can’t solve today’s problems with the same thinking that got you there” ……” insanity is continuing the same process over and over again and expecting different results” Albert Einstein

4. Us of Weapon Systems IT Acquisition Methods; Bureaucratic processes, upside-down incentives, redundant oversight, missing metrics (MOE, SLA) puts focus on compliance vs outcomes. Programs spending up to 25% on compliance without any reduction in risk. MilSpec Acquisition methods do not work for fast paced IT market. DODAF, JCIDS, NESI, LISI were designed for Weapons Systems (by FFRDCs), and have no track record for successful delivery of IT. Mis-application of FFRDCs and Defense Contractors who lack organic access to commercial best practices or low cost mechanisms to measure business value of commercial IT Solutions and Cloud Services, undermining ability to establish common & interoperable infrastructure services which accounts for 70% of every IT program buy. Concepts like SOA, Cloud Computing and Service Level Management cannot be embraced without a change in the above. Innovations and Best Practices Stifled: Contractors with IT buy/sell tech agreements or outcomes cannot objectively advise or firewall of OCI issues. Defense Industrial Complex suppliers are vested in costly design-to-spec development approaches that obscure commercial innovations. Traditional SIs are insulated from commercial IT innovations and industry best practices. Root Cause AnalysisSummary findings of 40 Studies & 20+ Program Failures across DHS, DoD and the IC

5. Unmet CCA and OMB’s IT Reform Directives Clinger Cohen Act Requires: • Streamline the IT Acquisition Process • Change business processes (BPR), not COTS • Favor COTS/OSS over custom development. • Build business case and acquire based objective assessment criteria • Use architecture for investment decisions • Adopt Commercial Standards and Best Practices OMB 25 Point Plan Requires: “IT Reform efforts must Align the Acquisition Process with the Technology Cycle. Point 13. Design and develop a cadre of specialized IT acquisition professionals . Point 14. Identify IT acquisition best practices and adopt government-wide. Point 15. Issue contracting guidance and templates to support modular development Point 16. Reduce barriers to entry for small innovative technology companies”

6. Unmet Defense Acquisition Reformsif implemented, would drive down IT cost overruns 2009 NDAA: ‘‘Implementing Management for Performance and Related Reforms to Obtain Value in Every Acquisition’’. Requires: (1) Determine clear performance metrics for specific programs from the start; (2) Foster an ongoing dialogue during the technology development process between the system developers and the warfighters; (3) Promote an open architecture approach that allows for more modularization of hardware and software; (4) Develop a plan for how to strengthen the IT acquisition workforce; (5) Implement alternative milestone decision points that are more consistent with commercial product development for IT; (6) Develop a process for competitive prototyping in the IT environment; (7) Develop a new test and evaluation approach that merges developmental and operational testing in a parallel fashion; (8) Place greater emphasis on the up-front market analysis; and (9) Conduct a rigorous analysis of contracting mechanisms and contract incentive 2010 NDAA Sec 804: “The Secretary of Defense shall develop and implement a new acquisition process for information technology systems. The acquisition process developed and implemented pursuant to this subsection shall, to the extent determined appropriate by the Secretary-- • be designed to include– 1) early and continual involvement of the user; 2) multiple, rapidly executed increments or releases of capability; 3) early, successive prototyping to support an evolutionary approach; and 4) a modular, open-systems approach” 2011 NDAA Sec 933: “The Secretary of Defense, in consultation with the Secretaries of the military departments, shall develop a strategy to provide for the rapid acquisition of tools, applications, and other capabilities for cyber warfare for the United States Cyber Command and the cyber operations components of the military departments. The Strategy shall include:” • Basic elements (1) An orderly process for determining and approving operational requirements. (2) A well-defined, repeatable, transparent, and disciplined process for developing capabilities to meet such requirements, in accordance with the information technology acquisition process developed pursuant to section 804 (3) The allocation of facilities and other resources to thoroughly test such capabilities. • Establish mechanisms to promote information sharing, cooperative agreements, and collaboration with international, interagency, academic, and industrial partners in the development of cyber warfare capabilities 2012 NDAA "Migration of Defense data and government-provided services from Department-owned and operated data centers to cloud computing services generally available within the private sector that provide a better capability at a lower cost with the same or greater degree of security.",

7. Critical Success Factors forAgile Acquisition & Assured Outcomes Capabilities established by the IT-AAC partnership as the ounce of prevention; • Embrace of Open and Agile IT Acquisition frameworks (per DSB report) already proven to meet challenges of the fast paced IT market (AAM is the only conforming todate) • Dynamic access to evolving commercial innovations, implementation best practices and lessons learned (CCA), outside the reach of the Defense Industrial Complex • A bottom up view of commercial capabilities that feeds the requirements process, (realm of the possible) to prevent over specification and costly custom development. • Means of deriving SLAs from both Measures of Effectives and Standards of Practice, critical to managed services, capability based acquisition and service level management. • Means of empowering and educating IT Program Management workforce via time proven expertise, tapping a wide range of gray beards with no inherent conflicts of interests. “To decrease risk in source selections, the DoD will follow proven commercial processes, increasing the emphasis placed on past performance and experience on prior government and commercial efforts in selecting IT providers.” DepSec Bill Lynn on Section 804 Implementation Plan

8. IT-AAC Centers of Excellence accelerating innovation and benchmarked best practices IT-AAC Partnership brings forth wide range of IT expertise need to mentor PMs: Governance and Oversight: how an enterprise supports, oversees and manages IT programs and on-going portfolio. SOA as defined in the commercial market is governance tool not technology. DoD5000 and BCL represent the current approaches. Decision Analytics: enables effective Program Management and Value Stream Analysis execution. As most of these sub-processes are designed to improve decision making, a relative new discipline has evolved (since 86), that addresses the human and cultural challenges in decision making. Decision Analytics is the discipline of framing the essence and success criteria of each gate in the acquisition lifecycle. It brings focus to the high risk areas of a program, and reduces analysis/paralysis. Capability Specification: Actionable requirements must be constrained by the realm of the possible. With pressures to do more with less, we must embrace mechanisms that force a relative valuation/impact of the gap/capability, with clearly defined outcomes Solution Architecture:This is one of the most critical elements of the acquisition lifecycle, as it should represent all stake holder agreements. The market embrace of SOA is not about technology, but a refocusing of the EA on service level management and data. A good architecture is a lexicon that links requirements, technologies and acquisition strategy. Technology Assessment: Understanding the limitation of technology early in the process is key. Without a clear view of the “realm of the possible” validated by real world results, we often find ourselves in high risk areas and over specification. Market research must be done early to help users constrain requirements and embrace the inherent business practices that codify. Recognizing that 70% make up of every IT application is vested in IT infrastructure (netcentric, cloud, SOA), it is critical to establish a common infrastructure/infrastructure standard by which all applications can share. The most prolific is ITIL to date. Business Case Analysis: Demonstrating the business value of technology investments, based on evidenced based research and lifecycle cost. This is a core requirement of Clinger Cohen Act. Procurement and Contracting: Software as a Service and SOA portend a new dynamic for acquisition of IT (health IT, cyber, business systems), that brings focus to Service Level Agreements (SLAs), Software as a Service (SaaS) and SL Management. If the previous activities do not directly feed the acquisition strategy or provide mechanisms for contractor accountability, all is lost.

9. IT-AAC vs Traditional Defense Advisors transforming and informing acquisition ecosystem

10. Past Performance = Assured OutcomesWhere AAM and IT-AAC have proven: better, faster, cheaper USAF: Streamlined COTS Acquisition Process. Applied to Server Virtualization. Contract Value: $500k Established optimal arch with ROI of 450% & $458 million savings Navy: Assessment of AFLOAT Program – CANES SOA & Security Strategy Contact Value: $350k Eliminated hi-risk Requirements by 23%, $100Ms in potential savings USAF: Full application of AAM Modules For eFOIA (KM) Contract Value: $150K Completed AoA, BCA, AQ Selection in just 4 months. BTA: Build out of AAM into BTA IT360, with two completed Pilots Contract Value: $300kM $300 million in potential savings with minimal investment USMC: Solution Architecture, AoA and BBA for Cross Domain, Thin Client Contract Value: $300k Greatly Exceeded Forecasted Saving in both analysis and acquisition GSA: Financial Mgt System consolidation using AAM. Contract Value: $500k Moved FMS from OMB “red” to “green”. Eliminated duplicative investments that saved $200M JFCOM: MNIS Evaluation of Alternatives for Cross Domain Solutions Contract Value: $350k Evaluated 100’s of Options in 90 days, enabling stake holder buy in and source selection. GPO: Developed Acquisition Strategy for Future Digital System FDSys Contract Value: $150k Led to successful acquisition and implementation on time, on budget and 80% cheaper than NARA RMS BTA: Apply AAM to complete AoA and BCA for DoD SOA Project Contract Value: $250k Reduced pre-acquisition cycle time and cost of Analysis by 80% (4 months vs 18)

11. Acquisition Assurance Method (AAM)Improving Decision Analytics and Measured Outcomes Across the Acquisition LifecycleAAM is the only Agile Acquisition Process Standard to comply with OMB’s directives and NDAA Section 804. AAM is managed as an Open Source process by the Interop. Clearinghouse, a quasi standards body. AAM is the result of an 8 year investment, in cooperation with AF A6, AF AQ, Navy SPARWAR, USMC, OMB, GAO, GPO, and BTA. The BTA’s Capability Assessment Method complimented both the DoD 5000 and BCL as a Decision Analytics tool.

12. IT-AAC Agile Acquisition Touch PointsMeasures and Aligns Business Needs with Proven IT Capabilities IT-AAC Communities of Practice Business Requirements & Capability Gaps • Mission • Need: • MoEs • Mission Prioritization • Constraints Value Stream Analysis SDOs/Labs/ Universities Innovators Vendors/ISVs Industry CxOs Biz Process Re-Engineering Prioritized Business Requirements Research, Testing Results Evidence Lessons Learned Innovations Measurable Outcomes Business Metrics Solution Exist? • Service Specification • Feasibility • SOA Attributes • SLAs • Shared Services Proven IT Solutions Y Align Proven Capabilities w/ business needs Knowledge Exchange N Knowledge Exchange Service Oriented Specs and SLAs Model New Solution Normalized Service Components Solution Set Evidenced-Based Assessment • Technology Fit/Finish: • Selection • Certification • Interop Spec • Openness Validated Past Performance Validated Acquisition Strategy, SLAs & Source Selection Criteria Vetted Solution Architecture Analysis of Alternatives Solution Architecture Validation and Demonstrations COTS Comparative Analysis, Evidence Y N

13. AAM Aligns IT with Business Needs increasing stake holder value, reducing risks Reference Models Associated Metrics BRM Business Drivers & Metrics (BRM/OV) Performance Metrics User/Integrator Best Practices Core Business Mission Objectives Business Processes & Infrastructure Business Driven Top Down Security Profiles Effectiveness/Efficiency AAM Service Component BRM Service Components & Metrics (SRM/SV) Appl Service Components Layer 1 Infrastructure Service Components Layer N SAIL Solution Frameworks Aligns with business needs Common Criteria Vendor Solution Templates Interoperability, Fit, Finish BRM Technical Solution & Metrics (TV, TRM) Application Layer 1 Common Infrastructure Layer M Secure Solutions

14. Acquisition Strategy Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Clear Problem Statement, Capability Gaps • RFI Assessment • Realm of the Possible • Measures of Effectiveness • Other data as Price lists • Determine Sponsor and Stake Holder representatives • Codify Business Problem statement • Validate Project Scope, Timeline, Outcomes • Collect and evaluate existing data from RFI responses and other sources • Deliver Project POAM • Establish Stake Holder Agreement and Success Criteria • Establish Measures of Effectiveness Critical Success Factors • Entry Criteria • Initial Data collection • Initial identification of Capabilities • Business Needs & Gaps • Exit Criteria - Outcomes • Approval of Project Plan • Approval of Business Problem and Outcome • Criteria: Adequacy of Capabilities or Plan for correction 0 months 1 month 2 months 3 months 4 months 13

15. Requirements Analytics Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Capture Problem Statement w/Sponsor • Establish Performance Measurements • Document Agency Services Baseline • Determine industry capabilities and metrics • Capture Function Capabilities • Determine level of granularity needed • Hold Requirements WGs w/Sponsor's Key Stakeholders • Publish Capability Analysis Report (CAR)(Requirements and their Justification) • Work papers on: • Justification of Requirements, & Capabilities • Problem Statement Validation Critical Success Factors • Entry Criteria • Approved Project Plan and POAM • Exit Criteria • Approval of the CAR by the Functional Sponsor Criteria: Adequacy of Capabilities or Plan for correction 0 months 1 month 2 months 3 months 4 months 14

16. Capability Development Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Analysis Group; Service Components reference model mapping to capabilities/requirements • Work papers on: • Results of the Market Survey • Standards of Practice • Industry Benchmarking Data • Standardized Vocabulary for describing service components and basis for establishing SLAs (not in scope) • Refine Capabilities into Service Component solution models (per OMB FEA-PMO) • Conduct Market Survey • Establish Service Component & Groupings • Review RFQ for adequacy of detail • If RFI responses lack depth or breadth, ICH will conduct Industry Outreach and Benchmarking • Construct Service Component Analysis Groups • PMO review Critical Success Factors • Entry Criteria • Approved CAR (Validated Capabilities) • Exit Criteria • Approval of Service Component by the PM Criteria: Adequate industry metrics or plan for correction 0 months 1 month 2 months 3 months 4 months 15

17. Performance & Risk Metrics Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Hold Functional WG w/Sponsor's Key Stakeholders • Develop Prioritization Weighting Scale • Team Normalized weighting of the Service Components • Document each weights rationale • Capability Prioritization Matrix • Work papers on: • Service Component Prioritization Scale • Rationale for each weight given (traceability) Critical Success Factors • Entry Criteria • Approved Analysis Groups, Service Components and Standards of Practice • Exit Criteria • Approved Capability Prioritization Matrix Criteria: Functional Sponsor Approval 0 months 1 month 2 months 3 months 4 months 16

18. Market Research Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination Activities Artifacts/Deliverables • Evaluate RFI Responses • Establish alternatives for the assessment • Establish Scoring WG team • Develop Scoring Plan • Score Alternatives + + • Perform Sensitivity Analysis on Scoring Results • Analyze results • Review AoA date points • Present Results to Functional Sponsor - May included Functional WG team • Analysis of Alternative (Compare New/Existing Solutions against Prioritized Capability) • Work papers on: • Scoring Plan • Scoring Rationale • Sensitivity analyses performed • Technology Maturity Assessment Critical Success Factors • Entry Criteria • Approved Capability Prioritization Matrix • Exit Criteria • Approval of Feasibility Assessment Report by DBSAE/ PMO Criteria: (1) Assessment Team agreement on the scores. (2) Reference material justifying scores 0 months 1 month 2 months 3 months 4 months 17

19. AoA and Business Case Economic Analysis Capability Analysis Feasibility Assessment Project Strategy Capability Prioritization Capability Determination 3 Activities Artifacts/Deliverables • Setup Main Cost Model • Determine the quantities and time frame to be Evaluate • "Setup Sub-Models for direct, indirect & migration cost + savings" • Determine Model's elements related to ROI • Determine the models for each alternative • Collect Data industry data and assumptions • Conduct TCO • Review of Economic Analysis Results • Present Results to Functional Sponsor - May included Functional WG team • Develop Economic Analysis Report • Economic Analysis Report • Solution Architecture • Documented CCA compliance • Work papers on: • Model Documentation • Documentation of each Alternative • Documentation on costs developed for the Mode; • Documentation of Industry Metric determined Critical Success Factors • Exit Criteria • Approval of Economic Analysis Report by Functional Sponsor Criteria: (1) Functional Sponsor Agreement (2) Reference material justifying cost models • Entry Criteria • Approved Feasibility Assessment, AoA 0 months 1 month 2 months 3 months 4 months DBSAE Assessment

20. In Review: A Proven Alternative“the ounce of prevention in these lean times…” • Open & Inclusive Structure (conforming to OMB A119 and NTTAA) • 501.C6 Non-Profit Research Institute: Conflict free support and secure services • Access to market innovations and lessons learned: Rapidly applied to the Government • 39 public/private partnerships, 3,000 industry fellows: Massive “think tank” • Top Secret Clearances, Access to tens of thousands of SMEs • Repeatable, Measurable, Standardized Agile Methods (AAM) • Acquisition Assurance Method (AAM) validates and aligns requirements, solution architecture & perform. metrics • Evidenced Based Research (EBR) validates needs, market capabilities while mitigating risk • Proven to reduce over specification, costly customization and lifecycle costs • Exceeds Agile criteria found in NDAA Sec804 and OMB 25 Point Plan • Conflict Free Studies, Assessments and Analysis Services • AoA, EoV, BCA (ROI) and Risk Assessments at a fraction of the cost and time • Reusable studies and benchmarks: COTS, Open Source, ERP, Infrastructure and SOA • LEAN Six Sigma, Solution Architecture, IT Acquisition, Portfolio Mgt, SOA Governance, ITIL/VCA • Assured Mission Outcomes, Low Cost: AF, Navy, USMC, BTA, JFCOM, GSA • Evidenced Based Research derived from real world results • Reduced acquisition time-line, measurable and predictable outcomes • Significant cost-avoidance and savings, reduced analysis/paralysis

21. FFP GSA Acquisition Assurance Offeringsvia Simplified Acquisition MethodMOBIS: GS10F0540N or Sched 70: GS35F0151M

23. IT-AAC Partner Capabilities and Expertiseenabling sound decisions • Non-profit free think tank composed of the worlds most respected public service institutes and domain experts not available through traditional contracting mechanisms • Innovation Clearinghouse and Knowledge Exchange that captures proven market innovations in an acquisition ready context (canonical form) • Benchmarked Best Practices and Lessons Learned (SOA, Cloud, IaaS, PaaS, SaaS, Web Services) provided by customers who share business value from real world implementation and testing results • Acquisition Decision Framework that pools and normalizes infrastructure requirements, architectures, tech assessments, performance metrics (SLAs) business case analysis, and evaluation criteria. • Leadership Roundtables and Educational Forums that provides a hype free interchange with government and industry leaders • Virtual Solution Architecture Integration Lab (SAIL) and Solution Architecture Working Groups (SAWG), that detail realm of the possible and pre-validate fit and finish of commercial solutions. • Acquisition Advisory Contract Vehicles that reduce time to market and accelerate acquisition outcomes "It is not a great mystery what needs to change, what it takes is the political will and willingness, as Eisenhower possessed, to make hard choices -- choices that will displease powerful people both inside the Pentagon and out” Defense Secretary Robert Gates

24. Infrastructure as a Service (IaaS) Software as a Service (SaaS) Platform as a Service (PaaS) ITILv3 Secure Cloud Computing Shared Services Architecture Service Oriented Architecture FEA-PMO Reference Models Service Level Management Performance Based Contracting Thin Client & Server Based Computing Cross Domain Solutions Large Data Management Federated Data XML/Object/Relational Hybrid Knowledge and Content Management (eFOIA) Portfolio Management Secure Information Sharing Identity Management and InfoSec Agile Development Tools IT-AAC Domains of Expertiseleveraging what works and proven in the market