The Journey to Level 4

1. The Journey to Level 4 Presented by: Ms. Jackie Langhout, Lead of the Software Engineering Process Group U.S. Army Aviation and Missile Command Software Engineering Directorate at the Huntsville SPIN meeting 12 April 2001

2. AMCOM SED Army Materiel Command Research, Development, and Engineering Center Mission - Provide Mission Critical Computer Resource (MCCR) expertise to support AMCOM weapon systems over their life cycle. - AMCOM focal point for MCCR - Acquisition and Technology - Post Production Software Support - Interoperability Engineering & Testing AMCOM Life Cycle Software Engineering Center Dept. of the Army Aviation & Missile Command Software Engineering Directorate (SED) AMCOM Software Engineering Directorate

3. SED Characteristics • 150 Government and 450 contractor personnel~ 300,000 sq. ft. facility with laboratories and space for tactical equipmentCustomers include Missile and Aviation PEOs, AMCOM Project Offices and Foreign Military Sales 15 – 20 software development or sustainment projects (~ 230 personnel) Artist rendering of SED Annex, Bldg. 6263 AMCOM Software Engineering Directorate

4. SED Project Characteristics • Type: • New Development - 40% • Sustainment – 60% • Applications: • Trainer/Simulator • Test Device/Tool • Command, Control & Communication • Tactical • Size of team: 4 – 25 people • Duration: 3 – 24 months • Product size: 400 – 160,000 LOC • Languages: 11 including Ada, C, Fortran. Project Team Project Lead Engineering Staff SQA CM AMCOM Software Engineering Directorate

5. SED Process Improvement Program • Based on the SEI’s Capability Maturity Model for Software.Monitored by the Quality Management Board (SED senior management).Facilitated by the Software Engineering Process Group (Government and contractor).Supported by the training program. Dir. & Dep. Dir. QMB Div. Chiefs SEPG TAC Project Teams Resources: SEPG = 9 people, 4 FTE MYs TAC = 5 people, 1.5 FTE MYs Level 1 Level 2 Level 3 Level 4 1990 1991 1994 1996 2000 AMCOM Software Engineering Directorate

6. Challenges to Reaching Level 4 • Understanding the CMM terminology. • Improving the metrics program. • Documenting the processes and procedures. • Training the personnel. • Maintaining focus during implementation. AMCOM Software Engineering Directorate

7. Challenge #1 • Challenge: Understanding the terminology in the CMM/SW Level 4 Key Process Areas. • Responses: • Researched and studied available material (i.e. Crosstalk, SEI web site) • Participated in conferences and symposiums (i.e. SEPG, STC, SEI) • Communicated with the SEI (i.e. Paulk, Carleton, Florac). • Interfaced with high maturity organizations (Air Force). AMCOM Software Engineering Directorate

8. CMM/SW Level 4 • Managing with facts and data. • Understanding how processes perform.Understanding the quality of the developed products.Using organizational business goals and projects’ issues to determine the quantitative analysis applications. AMCOM Software Engineering Directorate

9. Challenge #2 • Challenge: Improving the metrics program. • Responses: • Employed a full-time metrics analyst with a background in statistical process control. • Established organizational goals with senior management. • Aligned the metrics with the organizational goals and process (SEPH). • Automated the collection and reporting of metrics. AMCOM Software Engineering Directorate

10. Metrics Program • Metrics: effort, size, duration and defects. • Metrics Database contains: > 265 builds > 300 Peer Reviews > 250,000 effort hours > 455,000 lines of delivered code (11 languages) • Statistical Process Control (SPC) • applied to: • Productivity • Review (Scan) Rate for peer reviews • Defect Density for peer reviews AMCOM Software Engineering Directorate

11. Challenge # 3 • Challenge: Documenting the processes and procedures. • Responses: • Utilized an approach appropriate for SED. • Updated the SEPH, the SEPG Operations Handbook and other documents to include the Level 4 activities. • Incorporated the quantitative process management plans into the project plans. • Modified peer review procedures and forms to include the application of SPC. AMCOM Software Engineering Directorate

12. Standard Software Process • The SEPH (SED Engineering Process Handbook) • Contains processes, procedures and standards for new development, transition and sustainment projects. • Includes guidance for the collection of project metrics. • Tailoring of the SEPH is performed by the Project Team, reviewed by the SEPG mentor and approved by management. AMCOM Software Engineering Directorate

13. Challenge #4 • Challenge: Training the Level 4 process and procedures. • Responses: • Modified previously developed courses. • Developed and taught an SPC application course. • Minimized class durations and maximized course offerings. • Supplemented with mentoring from SEPG members. AMCOM Software Engineering Directorate

14. Training Program • Required training based on role performed. Training Database maintains records.Training schedule, data, charts available on-line.Management provided semi-annual progress reports. Training Plan Training Program Handbook Training Report Training Totals AMCOM Software Engineering Directorate

15. Challenge #5 • Challenge: Maintaining focus during implementation. • Responses: • Conducted regular SQA audits and an internal mini-assessment. • Reported readiness to senior management on a regular basis. • Utilized the email, newsletter and intranet as communication sources. • Held assessment preparation meetings with all projects. AMCOM Software Engineering Directorate

16. Assessment Details • 2 week CMM Based Appraisal for Internal Process Improvement • Certified Lead Assessor – Dave Zubrow of SEI • Assessment Team members represented 6 different organizations/companies • 54 Assessment participants • 18 projects • completed CMM Compliance Checklists • compiled Artifact Notebooks • participated in interview sessions • 4 Project Leads from diverse projects AMCOM Software Engineering Directorate

17. Maturity Profile Apr 2000 Level 4 SED Assessments Nov 1996 Level 3 May 1994 Level 2 Major Determinants in a Process Capability Rating Sept 1991 Level 1 • The Process is Documented • The Process is Taught • The Process is Used • The Process is Measured *From March 2001 SEI report, based on most recent assessment of 1012 organizations. AMCOM Software Engineering Directorate

18. Results • Increased productivity • Since 1995, productivity has doubled. • Peer reviews are 9 times more cost effective than testing. • High quality products • Post delivery defects are below 1 per KSLOC. • In past two years, a 10% decrease in the defect density of peer reviewed products. • Organizational growth • The number of software development/sustainment projects has increased by a factor of 4 since 1992. AMCOM Software Engineering Directorate

19. Keys to Success • Long term, consistent commitment by senior management. • Total staff (Government and contractor) involvement. • Establishment of organizational goals provided guidance for the metrics program. • Automation used to disseminate information and collect data. • Dialogue with SEI and high maturity organizations. AMCOM Software Engineering Directorate

20. Next Steps • Continue to improve our software engineering processes (tools/technology evaluation and insertion, defect analysis).Transition to the CMMI for System and Software Engineering.Implement Personal Software Process/Team Software Process (PSP/TSP) on selected project(s). AMCOM Software Engineering Directorate