LOG 203 (R&M) - LOG 103 (RAM) Migration Prepared for the Life Cycle Logistics (LCL)

1. LOG 203 (R&M) - LOG 103 (RAM) Migration Prepared for the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) Presented by Patrick M. Dallosta, CPL Performance Learning Director LMI Government Services McLean VA 15 January, 2010 1

2. Agenda • Change Drivers • LOG 203 Status • LOG 103 Content • Migration Plan 2

3. Recent Legislation/updated DoD Policy has significantly strengthened DOD Reliability, Availability and Maintainability (RAM) Policy, Systems Engineering/Reliability/Test & Evaluation processes and organizations. Life Cycle Logistics DAWIA certification path reflects new course requirements in the area of RAM and Supportability LOG 103, “Reliability, Availability and Maintainability” Upgrading of existing LOG 203 LOG 211, “Supportability Analysis” New curriculum development Expect great cross-functional interest as RAM / Supportability is reconstituted in program offices – a “pendulum swing” back to government resulting from the aftermath of Acquisition Reform Change Drivers 3

4. FY13 LCL Certification Construct Level III Certification Level I Certification LOG 101 Acquisition Logistics Fundamentals ACQ 101 Fundamentals of Systems Acquisition Management P LOG 340 Performance Based Life Cycle Sustainment 30 hrs, on-line P LOG 102 Systems Sustainment Management 8.5 days classroom 2 Continuous Learning Modules: PBL & Designing for Supportability LOG 350 Enterprise Life Cycle Logistics Management 28 hrs, on-line P 25 hrs, on-line LOG 103 Reliability, Availability & Maintainability 1-3 hrs ea, on-line 8.5 days classroom 26 hrs, on-line • Knowledge based • GS 5-9 & E7-O3 • Case/scenario based • GS 13-14 & E7-O5 Level II Certification NOTE: There would still be NO prerequisites for LOG235 LOG 200 Intermediate Acquisition Logistics P ACQ 203 Intermediate Systems Acquisition LOG 235 Performance Based Life Cycle Product Support 35 hours on-line P LOG 201 Intermediate Acquisition Logistics 4.5 days classroom 40 hours, online P P ACQ 202 Intermediate Systems Acquisition 4.5 days classroom LOG 211 Supportability Analysis P LOG 206 Intermediate Systems Sustainment 35 hours, online 4.5 days classroom xx hours on-line • Application/case based • GS 9-12 & E7-O4 Level I “Core Plus” Courses & CL Modules (See DAU catalog for details) Level II “Core Plus” Courses & CL Modules (Includes LOG 204 CM Course & new LOG 2xx Tech Data Mgt Course) Level III “Core Plus” Courses & CL Modules (See DAU catalog for details) New Cert Courses 4 P = Prerequisite

5. LOG 203 Course Objective • Understand the relationship between Acquisition Logistics and Reliability & Maintainability (R&M) • Evaluate the impact of R&M decisions • Knowledge of basic R&M concepts, terminology engineering practices and math • Discussion of Practical applications 5

6. Crosswalk to LCL Competencies • Logistics Design Influence • LOG 101, LOG 103 (203), LOG 200, LOG 201, LOG 350 • Integrated Logistics Support (ILS) Planning • LOG 101, LOG 200, LOG 201, LOG 210, LOG 350 • Product Support & Sustainment • LOG 102, LOG 206, LOG 235, LOG 236, LOG 350 • Configuration Management • LOG 204, LOG 206 • Reliability & Maintainability Analysis • LOG 103 (203),LOG 200, LOG 201 • Technical/Product Data Management • LOG 204, LOG 206 • Supportability Analysis • LOG 103(203), LOG 200, LOG 201, LOG 210 6

7. LOG 203 Overview • Assignment specific course for Reliability & Maintainability • No prerequisites • Distance learning format / Only 1 Section of LOG 203 • LOG 203 metrics consistently meets performance expectations: • 3-fold increase in student throughput since 2006 • Key Personnel: • Performance Learning Director: Tom Simcik, LCIC-LOG • Course Manager: Richard DiLorenzo, Mid-West • Course Manager Regional Reps : Hal Ernest, South Phil Littrell, Mid-Atlantic Great Job, Tom and Team!! 7

8. LOG 203 Metrics & Demographics No FY 10 Data 6.00 5.89 5.95 6.12 6.08 5.98 6.02 6.18 5.85 5.89 5.82 5.99 5.64 5.74 5.74 5.91 * Data (from Data Mart) as of 31 Dec 2010 8

9. LOG 203 Metrics & Demographics (cont.) * as of 14 Jan 2010 9

10. Current and Proposed LOG 103 Lessons • Introduction to LOG 103, Reliability, Availability & Maintainability • Historical Roots of Reliability, Availability & Maintainability • Impact of Acquisition Reform on Reliability, Availability & Maintainability • Mandatory Sustainment Metrics KPP/KSAs • Legislative/Policy Reliability, Availability & Maintainability Initiatives • The Systems Design Operational Model (SDOE) • The Systems Engineering – Design Interface • Software Reliability, Availability & Maintainability • Designing and Enabling Reliability, Availability & Maintainability • Modeling & Simulation of Reliability, Availability & Maintainability • Supportability Elements, Practices and Outputs • Impact of RAM on Mission Success and Logistics • Translating Operational Requirements to RAM Requirements • RAM Growth Programs, Test and Evaluation and Risk Reduction • Impact of R&M on Manufacturing 10

11. On-going LOG 201-Log 103 Conversion Actions Convert current LOG 203 to DAU standard application software Update basic course information, e.g., re-title LOG 103 Deployment of LG 103 Certification Level Course 15 Mar 10 Additional LOG 103 Development New LOG 103 competencies/proficiencies identified 30 Apr 10 Draft curriculum developed 31 Jul 10 Draft curriculum review 31 Aug 10 Curriculum finalized 30 Sep 10 Programming /Test 31 Dec 10 Instructor Pilot/Student Validation (Jan 11) 31 Jan 11 Deployment 15 Mar 11 Migration Path Ahead 11

12. Questions & Answers LOG 203 – LOG 103 Migration Thank you for the Opportunity to Serve the Life Cycle Logistics (LCL) Functional Integrated Product Team (FIPT) 12

13. Back-up!!!

14. Lesson 1: Impacts of R&M on Logistics Recognize how R&M affects support costs Recognize how R&M affects Total Ownership Costs List 4 key elements in the definition of reliability List 2 MTB_s that are logistics reliability parameters Lesson 2: Impacts of R&M on Missions Distinguish between latent failures, random failures and wearout Identify 3 measures of combat capability that are enhanced by R&M Distinguish between reliability and maintainability Lesson 3: Operational & Contractual R&M Requirements List 3 methods for converting operational R&M requirements into contractual R&M requirements List 3 approaches for assessing R&M of Commercial and Non-Developmental Items Proposed LOG 103 Learning Objectives/Outcomes

15. Lesson 4: Providing Insight into Designing for Reliability Identify the reason that failure rates are easier to deal with than MTBFs Distinguish between mission reliability and logistics reliability Identify the aim of Reliability Centered Maintenance Analysis Lesson 5: R&M and Risk Reduction Recognize the Test-Analyze-and Fix process of achieving reliability growth Recognize the concept of decision risk in reliability qualification test (RQT) Lesson 6: Impacts of R&M on Manufacturing Identify the effect that latent defects can have on missions and logistics How to recognize a manufacturing defect from field data 3 lines of defense to keep manufacturing from producing latent defects Current Learning Objectives (Continued)

16. Lesson 4: Providing Insight into Designing for Reliability Identify the reason that failure rates are easier to deal with than MTBFs Distinguish between mission reliability and logistics reliability Identify the aim of Reliability Centered Maintenance Analysis Lesson 5: R&M and Risk Reduction Recognize the Test-Analyze-and Fix process of achieving reliability growth Recognize the concept of decision risk in reliability qualification test (RQT) Lesson 6: Impacts of R&M on Manufacturing Identify the effect that latent defects can have on missions and logistics How to recognize a manufacturing defect from field data 3 lines of defense to keep manufacturing from producing latent defects Proposed New Learning Objectives

17. Lesson 5: Impact of Legislation and DoD Policy on Reliability, Availability and Maintainability Discuss USD(AT&L) RAM Policy Memorandum of July 08 and its impact on systems engineering, reliability/logistics and test & evaluation, with emphasis on reliability growth programs Discuss impact of the Weapons System Acquisition Reform Act of 2009 and updates to DoDI 5000.02 on the Defense Acquisition Management System, systems engineering processes, sustainment and governance Lesson 6/1: Impacts of RAM on Logistics Recognize how RAM affects support costs Recognize how RAM affects Total Ownership Costs List 4 key elements in the definition of reliability List 2 MTB_s that are logistics reliability parameters Discuss the definitions of availability Lesson 7/2: Impacts of RAM on Missions Distinguish between latent failures, random failures and wear out Identify 3 measures of combat capability that are enhanced by RAM Distinguish between reliability and maintainability Proposed LOG 103 Learning Objectives/Outcomes • Lesson 1: Introduction to LOG 203 Reliability, Availability and Maintainability (RAM) • Identify the role of LOG 103 within the LCL and other DAWIA Certification Paths • Identify the relationships between LOG 103 and other certification courses • Identify the role of Reliability, Availability and Maintainability(RAM)as a key part of the systems engineering process • Identify the role of Reliability, Availability and Maintainability(RAM) in minimizing the need for logistics • Lesson 2: Historical Roots of Reliability, Availability and Maintainability (RAM) • Recognize Early Commercial/DoD Reliability Analyses, Tools and Outcomes • Recognize development and use of DoD Reliability, Availability and Maintainability processes, specifications and documentation • Recognize the impact of RAM on DoD Systems performance, sustainment and costs • Lesson 3: The Impact of Acquisition Reform on Reliability, Availability and Maintainability • Discuss the scope and impact of the Perry Memorandum • Discuss the Loss of workforce and skilled RAM personnel • Discuss the Discontinuance of Military Standards and Specifications and its impact • Discuss the Impact of the use of Commercial-Off-The-Shelf on system reliability and performance • Discuss the decrease in effectiveness and suitability as determined by operational Test and Evaluation • Lesson 4: Mandatory Sustainment Materiel Availability Key Performance Parameter and Supporting Key System Attributes of Materiel Reliability and Ownership costs • Discuss the origins of the Material Availability KPP, the Material Reliability KSA and the Ownership Cost KSA • Identify the impact of the KPP/KSA on the requirements management process • Discuss the development of the KPP/KSAs within the JCIDS process • Discuss the enablers to materiel readiness that impact RAM, e.g., RCM, CBM, diagnostics/prognostics

18. Lesson 8/3: Operational & Contractual RAM Requirements List 3 methods for converting operational RAM requirements into contractual RAM requirements List 3 approaches for assessing RAM of Commercial and Non-Developmental Items Lesson 7: The Systems Design and Operational Effectiveness (SDOE) Model Discuss the SDOE model and domains Discuss specific SDOE engineering outputs and their impact on performance, RAM, and sustainment Lesson 8/4: Providing Insight into Designing for RAM Discuss the concept of design interface Discuss the elements of supportability Discuss specific Reliability Availability & Maintainability trade space considerations Discuss common methodologies and analytic tools Identify the specific analyses to be conducts, e.g., block diagram development, allocation and prediction, failure modes and effects analysis, maintenance task analysis Identify the reason that failure rates are easier to deal with than MTBFs Distinguish between mission reliability and logistics reliability Current Learning Objectives (Continued) • Identify the aim of Reliability Centered Maintenance • Identify the aim of Condition Based Maintenance • Identify the aim of diagnostics and prognostics • Lesson 9/5: RAM Growth Programs and Risk Reduction • Discuss requirements for reliability management programs • Discuss requirements for reliability growth test programs • Discuss best practices • Recognize the Test-Analyze-and Fix process of achieving reliability growth • Recognize the concept of decision risk • Lesson 10/6: Impacts of RAM on Manufacturing • Identify the effect that latent defects can have on missions and logistics • How to recognize a manufacturing defect from field data • 3 lines of defense to keep manufacturing from producing latent defects • Lesson 6/3: Operational & Contractual RAM Requirements • List 3 methods for converting operational RAM requirements into contractual RAM requirements • List 3 approaches for assessing RAM of Commercial and Non-Developmental Items • Lesson 7/4: Providing Insight into Designing for Reliability • Identify the reason that failure rates are easier to deal with than MTBFs • Distinguish between mission reliability and logistics reliability • Identify the aim of Reliability Centered Maintenance Analysis • Identify the aim of Condition Based Maintenance • Identify the aim of diagnostics and prognostics • in reliability qualification test (RQT) • Lesson 9/6: Impacts of RAM on Manufacturing • Identify the effect that latent defects can have on missions and logistics • How to recognize a manufacturing defect from field data • 3 lines of defense to keep manufacturing from producing latent defects

19. Defense Science Board Principal Findings and Recommendations

20. Systems Failing Reliability During Operational Testing (OT)

21. Recent Logistics Management Institute (LMI) and Army studies investigated relationship between reliability and Life Cycle Cost (LCC). Both studies concluded that reliability investment during design and development reduces support cost LMI study concludes that: 30-60% reduction in support cost is achievable Reacting to lower reliability after IOC (vice during development) results in significant support cost increases OSD CAIG study shows that support costs are two-thirds or more of total system cost DoD should address acquisition and LCC strategies concurrently Support cost bill payers should participate in requirements definition and trade studies during development Impact of Low Reliability on Life Cycle Cost (LCC) Upfront Investment in Robust RAM Effort Emphasizing Reliability Growth Reduces Life Cycle Costs

22. Acquisition Reform implementation had a detrimental impact on RAM practices With some exceptions, the practice of reliability growth methodologies was discontinued during System Design and Development (SDD) and deferred until production Relevant military specifications and standards were cancelled and not, in all cases, replaced with industry standards The technical/managerial workforce was reduced in most government program offices and test organizations RAM shortfalls are frequently identified during DT, but program constraints (schedule and funding) often preclude incorporating fixes and delaying IOT&E In some instances, programs had such serious RAM concerns that they were precluded from proceeding to production until the problems could be corrected. Improved RAM decreases life cycle costs and reduces demand on the logistics system Reliability Availability & Maintainability (RAM) Findings

23. “…the single most important step necessary to correct high suitability failure rates is to ensure programs are formulated to execute a viable systems engineering strategy from the beginning, including a robust RAM program, as an integral part of design and development. No amount of testing will compensate for deficiencies in RAM program formulation. To this end, the following RAM-related actions are required as a minimum:” Identify and define RAM requirements during the Joint Capabilities Integration Development System (JCIDS) process and incorporate them into the Request for Proposal (RFP) and as a mandatory contractual requirement During source selection, evaluate the bidders’ approaches to satisfying RAM requirements Ensure flow-down of RAM requirements to subcontractors Make RAM, to include a robust reliability growth program, a mandatory contractual requirement and document progress as a part of every major program review Develop a military standard for consistent RAM development and testing that can be readily referenced in future DoD contracts Ensure an adequate cadre of experienced RAM personnel are part of the Service acquisition and engineering office staffs USD(AT&L) Memo “RAM Policy” July 08

24. “…the aggregate lack of process guidance due to the elimination of specifications and standards, massive workforce reductions in acquisition and test personnel, and the high retirement rate of the most experienced technical and managerial personnel in government and industry has a major negative impact on DoD‘s ability to successfully execute increasingly complex acquisition programs. Major personnel reductions have strained the pool of experienced government test personnel A significant amount of developmental testing is currently performed without the needed degree of government involvement or oversight and in some cases, with limited government access to contractor data The current trend away from governmental involvement in DT makes operationally oriented testing in some areas during development even more difficult to accomplish Roles and Responsibilities of Government Test & Evaluation (T&E) Organizations

25. LCL Course Development & Revision Roadmap Note: FY10 Deployment of LOG 103 & LOG 206 are a DAU Performance Plan Target under Task 1.1.1 (Develop and deliver high-quality learning assets and services to our customers).