Earned Value Executive Overview

1. Earned Value Executive Overview George Stubbs, CBT Workshop

2. About George Stubbs the Presenter • Over 45 years Program Management Experience. • 5 Years United States Air Force. • 6 Years Raytheon. • 8 Years Boeing. • 7 Years Martin Marietta. • 9 Years Independent Consultant. • 12 Years Senior Partner The CBT Workshop • The CBT Workshop. • A leading supplier for affordable skilled Earned Value Management System (EVMS) resources specializing in EVMS services, training, and tools support. The CBT Workshop www.cbtworkshop.com

3. Earned Value Executive Overview • Why Earned Value. • Earned Value Concept. • Earned Value Example. • What Makes up an Earned Value System. • Earned Value Software. • Earned Value Documentation. • Who is the Key to Earned Value Success. • Earned Value Training. • Major Earned Value Events. • The Integrated Baseline Review (IBR). • Validation/Certification. • The Earned Value Gap Analysis. • Some Bad Earned Value Events. The CBT Workshop www.cbtworkshop.com

4. Why Earned Value • DOD Contractual Requirement. • Earned Value Management System (EVMS) in compliance with guidelines in ANSI/EIA-748 is required on all cost or incentive contracts equal to or greater than $20M. • A formally validated and accepted EVMS is required for cost or incentive contracts equal to or greater than $50M. • EVM may be imposed on contracts less than $20M as a risk-based decision of the program manager. • Other Government Department and Agencies are requiring EV. • Some Contractors consider it best practice. The CBT Workshop www.cbtworkshop.com

5. The Earned Value Concept • Break work into measurable segments. • Segments are called work packages (WP). • Assign a cost per segment/WP. • Assign a duration per segment/WP. • Develop a measure of performance. • Collect actual costs (from accounting). • Collect performance (measure completion or progress). • Status schedule. • Determine Earned Value. • Compare actual costs verses actual performance. The CBT Workshop www.cbtworkshop.com

6. Earned Value Example • Build 12 miles of road for $12M in 12 Months • What if: • 1 mile in 1 month for $1M. • On schedule, on cost and will finish on target if progress continues. • 1 mile in 2 months for $2M. • One month behind schedule, $1 over cost and will finish in 24 month at a cost of $24m if progress continues. • 1.5 miles in 1 month for $1.25M. • .5 month ahead of schedule, $.25M cost under run and will finish in 8 (12/1.5) months for at total cost of $10M (1.25X8) if progress continues. The CBT Workshop www.cbtworkshop.com

7. What makes up Earned Value Systems • Software. • Accounting System. • Scheduling System. • Earned Value Engine. • Analysis tools(optional). • Documentation. • System Description Document. • Procedures (desktops). • Directives. • Training. • General EVMS. • CAM Specific. • Executive EV Overview. • Software tools. The CBT Workshop www.cbtworkshop.com

8. Earned Value Software • Accounting System. • Cost Point, In house developed, SAP. • Cost collection at the WP level, totals at the program level. • Scheduling System. • MS Project, Open Plan, Primavera. • Earned Value Engine. • Cobra, MPM, DekkerTrakker, Primavera Cost Manager, EcoSys, Artemis Costview. • Analysis tools(optional). • wInsight, Steel Ray, Acumen Fuse, Risk Plus, PM Compass, Active Risk Manager, The CBT Workshop www.cbtworkshop.com

9. Earned Value Documentation • System Description Document (SDD). • Describes how the contractor intends on complying with ANSI EIA 748-B. • Details how each of the 32 criteria will be addressed. • Procedures (desktops). • Explains in detail how company personnel will apply the companies SDD to specific EVM processes. • Directives. • Corporate/Company/Program direction that implements EVM or portions of EVM. The CBT Workshop www.cbtworkshop.com

10. The CAM is the Key to Earned Value Success • The Control Account Manager (CAM)will make or break a EV program. • The CAM sets, controls, and reports all performance criteria. • Is the daily face the customers team interacts with. • Are always interviewed by the customer for IBRs and Validations. • Knowledgeable CAMs can offset marginal EV systems. • Poor CAMs will cause a great EV system to fail. • The better the CAMs are, the better your EV system will perform.

11. Earned Value Training • General EVMS. • Basic EVMS training to all personnel on the EV program. • CAM Specific. • Detailed training to all Control Account Managers on how to perform EV. • Schedulers and Finance Analysts. • Detailed training on what the customer expects in their reports and on analysis tools used to review the submitted data. • Executive EV Overview. • EV overview training for all company executives in the EV program reporting chain and should include all supporting departments. • Software. • Hands on software and EV training of all personnel using the EVMS tool sets. The CBT Workshop www.cbtworkshop.com

12. Major Earned Value Events • Periodic Reporting. • Contract Performance Report (CPR)(monthly). • Schedule Reporting (monthly). • Other customer required (usually monthly). • Contract Funds Status Report (CFSR)(usually quarterly). • Milestones. • Integrated Baseline Review (IBR) (90 to 120 days after contract award). • Validation/Certification (after IBR and when required by customer). • Other. • Earned Value Gap Analysis. The CBT Workshop www.cbtworkshop.com

13. Integrated Baseline Review (IBR) • A formal review conducted by the customer. • At the contractors site. • Within 6 months of contract award. • Verifies the technical and cost content of the Performance Measurement Baseline (PMB). • Confirm the program baseline. • Review actual work performed. • Review the latest Revised Estimate to Complete (ETC). • Identification of Any Risks or Concerns. • Review the Earned Value Management System (EVMS). The CBT Workshop www.cbtworkshop.com

14. What is a EVMS Validation/Certification? • Also called Compliance review. • Required for all companies with more than $50 M in EVMS contracts. • Detailed review of all EVMS related items. • Similar to a IBR. • Concentrates on the CAMs and processes. • Results in a “Compliant” EVMS. • Single Validation required for each business entity. • Review and verify prior CARs, DRs and other EV actions.

15. What Does Not Passing Mean? Potential of extra withholding on payments that could be as much as 10% additional. Disadvantage in bidding future EVM contracts. Potential for additional oversight on existing EVM contracts. Brings in question the Companies ability to control and manage multi million dollar Programs.

16. Earned Value Gap Analysis • A review of a contractor’s internal policies and procedures to gage their compliance with all 32 guidelines called out in by ANSI/EIA-748. • Answers the following four key questions: • How does the existing management processes comply with ANSI/EIA-748? • Are there policies and procedures in place and are personnel following the processes? • Do the supporting program management systems provide the data required to support ANSI/EIA-748? • If any of the above does not support ANSI/EIA-748, what must be done to bring them into compliance? • All Gaps must be resolved before the contractor will have a compliant EVMS.

17. Some Bad Earned Value Events • A-12 • Based on the contractor’s Earned Value analysis that showed ‘catastrophic’ cost and schedule problems on this complex, multi-billion $ fixed price incentive contract, the then Sec. of Defense Dick Cheney withdrew his support for the program. Program litigation continues today. • Bell Helicopter • Previous estimates for the System Development Demonstration portion of the ARH program had grown from $210 million to over $300 million. • The Army filed a Nunn-McCurdy cost and schedule breach on July 9, 2008, when new cost estimates showed a 40% cost increase above initial estimates. • The cancelation was the result of excessive costs of the program which had increased over 70 percent with an estimated per-unit cost of US$14.5 million, up from US$8.5 million. • The Navy pulled Bell’s certification affecting its standing in the Cobra/Huey program. The company was eventually recertified, but only after great effort and expense. • Lockheed F-35 • The Pentagon has suspended its certification of Lockheed's Earned Value Management System. • The company’s system was deficient in 19 of 32 areas. The CBT Workshop www.cbtworkshop.com

18. EVM Works When Management embraces it. When the Program uses it to manage. When all Program levels understands how it works. When CAMs are rewarded for their additional work. Being a CAM is a “Stepping Stone” to becoming a Program Manager.

19. EVM does not work When Management pays lip service to it. When the Program fights it. When Management focuses on metrics not performance. When program levels do not understand how it works and what it does. When Performance or Actuals are tailored to fit desired metrics. When CAMs are not rewarded for their additional effort. When Management makes EVMS related decisions without feed back from program.

20. What is an Earned Value Validation and How is it Accomplished ? George Stubbs, CBT Workshop

21. Earned Value Validation • What is a EV Validation. • How is a Validation Done. • Review Objectives. • How is Validation Review Accomplished. • What Not Passing Means. • EVM Horror Stories. • Review Team Actions. • What the Contractor Needs. • Actions Required. • Artifacts Required. • When EV Works And When is Does Not.

22. What Is A EV Validation? • Required for all companies with a EVMS DOD cost plus contract over $50M . • Validation is achieved by conducting a formal review of the processes defined and used by the contractor to comply with the EVMS guidelines in ANSI/EIA-748. • It determines that the contractor is using the system as one of its primary program management processes. • That the contractor has properly implemented the system on the contract. • That the contractor is using the data from its system in reports to the government. • Single Validation required for each company. • Results in a ‘Compliant” EVMS. • An EVMS Compliance review is the same as a Validation review.

23. How is a Validation Done? • A formal recognition of certification by an independent party that a company’s EVMS meets the guidelines in ANSI/EIA-748. • A formal Government review conducted at a contractor’s facility to assess the contractor’s proposed EVMS compliance with ANSI/EIA-748. • A recurring process by an independent party, normally DCMA, assessing the continuing compliance of the company’s EVMS with ANSI/EIA-748 and the company’s written system documentation.

24. Review Objectives • Evaluate management system capabilities against ANSI/EIA-748. • Assess the description of the management system to determine if it adequately describes the management processes demonstrated during the review. • Evaluate the application of the management system on the contract being reviewed.

25. How is Validation Review Accomplished? • The VR begins as soon as practicable following the implementation of the EVMS and an IBR review. • The review consists of system documentation reviews, data traces, and interviews with contractor personnel. • The contractor’s EVMS is assessed against each guideline contained in ANSI/EIA-748. • The review should be organized to follow the system approach taken by the contractor, i.e., orientation by ANSI/EIA-748 grouping or a process approach.

26. What Not Passing Means • Potential of extra withholding on payments that could be as much as 10% additional. • Disadvantage in bidding future EVM contracts. • Potential for additional oversight on existing EVM contracts. • Brings in question the Companies ability to control and manage multi million dollar Programs.

27. EVM Horror Stories Lockheed Martin Aero • The 2007 DCMA EVMS compliance review found that LM-Aero is not following, nor consistently applying, EVMS during the execution of DoD Programs. • The findings of the review indicate that, the government is not obtaining useful program performance data to anticipate and mitigate program risks. • Key EVMS processes and procedures are below standard and do not provide the definition and discipline to properly plan and control complex programs. • Documented findings indicate that the Control Account Managers (CAM) can not satisfactorily demonstrate that they understand and use documented EVM processes, procedures and tools to manage their work.

28. LM-Aero Results • For all new contracts include a contract clause for withholding up to 5-10 percent from supplier payments for failure to adequately implement and maintain a validated EVM System. • For existing contracts, add a provision within the program award fee plan to establish similar incentives. • Lockheed Martin should review each LM-Aero location to ensure that the LM-Aero EVMS is properly implemented and that LM-Aero Fort Worth, TX findings are not systemic through out the LM-Aero division.

29. EVM Horror Stories Bell ARH Program • Lost EVMS compliance. • Payments on all CPFF reduced from 80% to 70%. • Company was under pressure to get compliant. • New EVMS program (ARH) awarded . • Mgmt suppressed schedule slips. • Prototypes were delivered behinds schedule. • Design changes impacted costs. • Mgmt suppressed cost impacts. • CEO fired . • Program canceled. • Recently completed positive DCMA audit.

30. Review Team Actions • Review the documentation that establishes and records changes to the baseline plan for the contract. • Review of the reporting of cost and schedule performance against the baseline plan, along with appropriate analyses of problems and projection of future costs. • A trace is conducted to summarize the cost/schedule performance data from the lowest level of formal reporting (normally the control account level) to the external performance measurement report. • Interviews with a selected sample of CAMs, functional and other work teams, and PMs to verify that the contractor's EVMS is fully implemented and being used in the management of the contract. • An exit briefing covering the team's findings.

31. What the Contractor Needs • The contractor should have a current, approved written system description available. • Applicable desktop procedures need to be available at the contractor’s operating levels as necessary to demonstrate a consistent approach. • The contractor should make documents used in the contractor’s EVMS available to the team. • The documentation needs to be current and accurate. • The contractor demonstrates to the team how the EVMS is structured and used in actual operation.

32. Actions Required • Develop a overview briefing by to familiarize the review team with the proposed EVMS. • The overview should identify any changes which have occurred since the most recent Progress Assistance Visit. • Have available the documentation that establishes and records changes to the baseline plan for the contract. • This includes work authorizations, schedules, budgets, resource plans, and change records including management reserve and undistributed budget records. • Develop CAM notebooks • Insure that all notebook data is current and that the CAM is familiar with how to find all data.

33. Actions Required (cont) • Develop a trace (storyboard) to summarize the cost/schedule performance data from the lowest level of formal reporting (normally the control account level) to the external performance measurement report. • The purpose of this activity is to verify the adequacy of the control aspects of the system and the accuracy of the resulting management information. • Train and perform mock interviews with CAMs, functional and other work teams, and PMs to insure that they understand the contractor's EVMS and that it is fully implemented and being used in the management of the contract.

34. Artifacts Required • The System Description. • Desktop Procedures. • CAM Notebook. • Storyboards and supporting documentation. • Prior IBR action items and their completion documentation. • Other past customer reviews as applicable.

35. The System Description • The EVM System Description provides a description of the system and the contractor’s comprehensive plan for meeting the guidelines in ANSI/EIA 748-A. • The document provides an understanding of each activity required to meet the EVMS standard. • The document should briefly, but comprehensively, present the contractor’s approach and schedule of internal activities to demonstrate that the system and processes meet the guidelines in the standard. • The description should include the contractor’s plan for implementation of, and activities leading up to, compliance/validation review by the Customer. • The document should also have a compliance matrix that shows the System Description reviewer where each section complies with ANSI/EIA-748.

36. Desktop Procedures • Every EV process should have a documented procedure. • Gives everyone a proven EV process to follow. • Insures compliance with the requirement of the guidelines in ANSI/EIA-748. • Gives the review team a road map to follow in the application of the specific EV process.

37. Typical Procedures • Project Startup • Organizational Breakdown Structure • Work breakdown Structure and Dictionary • Project Authorization Document • Budget Authorization Document • Work Authorization Document • Contract Brief • Responsibility Assignment Matrix • Opening Charge Numbers • Baselining Schedules • Statusing Schedules • Schedule Tool Integration • New or Change IMS • Time Phased Budget • Program Logs • Management Reserve • IMP-IMS Generation • Training • Desktop Metrics • Manager Review • Accumulate Actuals • WBS Maintenance • CA Monthly Checklist • Customer Reporting • Internal Reporting • Reading Reports • CPR Data Validation • Accumulate Budgets • Accumulate Status • Retroactive Adjustments • Estimate at Completion • Variance Thresholds • Variance Analysis Reporting • Corrective Action Log • Baseline Change Request • Baseline Change Request Log • Standards and Conventions • Metric Surveillance Report • Monthly Production Calendar • CAM Notebook Guide

38. The CAM Notebook • The Control Account Manager (CAM) notebook is the key document for assisting the CAM in the integration and management of the control account. • The CAM notebook enables “single thread” analysis of the technical scope, integrated master schedule (IMS), resource loading profile, earned value (EV) performance data, and subcontractor costs. • An accurate, current, and complete CAM Notebook is critical to the successful management of the contracted work. • The review of the CAM Notebook and identification of the resulting risks are key parts of the IBR process to verify and validate the technical, cost and schedule baselines.

39. CAM Notebook Contents • Organization Documentation. • Integrated Master Plan (IMS). • Statement of Work (SOW). • Work Breakdown Structure (WBS) and WBS Dictionary. • Work Authorization Document (WAD). • Control Account Plan (CAP). • Basis of Estimate and other cost baseline supporting information (BOE). • Integrated Master Schedule (IMP). • Earned Value Performance Report (CPR). • Subcontractor and Material Documentation. • Risk and Opportunity Management Documents.

40. Storyboards • Storyboards are a map of the EV process being reviewed. • They show monthly processes. • They detail the EV paperwork process from bid submittal through contract award and on to close out. • They give the viewer a step by step flow of the data used in the EVMS. • They should tie to the System Description and the desktop procedures. • Actual program documents should be used.

41. Storyboard of the CPR Flow

42. Mock CAM Interview • Provide CAM with Typical Questions Asked by Reviewers. • Have CAM Develop Briefing Covering Most Common Questions Asked. • Review All Contents of the CAM Notebook. • Have CAM Brief the Mock Reviewer. • Have the Reviewer Ask Some Questions Not Covered by the CAM’s Briefing. • Performed by someone not known to the CAM. The CBT Workshop www.cbtworkshop.com

43. EVM Works • When Management embraces it. • When the Program uses it to manage. • When all Program levels understands how it works. • When CAMs are rewarded for their additional work.

44. EVM Does Not Work • When Management pays lip service to it. • When the Program fights it. • When Management focuses on metrics not performance. • When program levels do not understand how it works and what it does. • When Performance or Actuals are tailored to fit desired metrics. • When CAMs are not rewarded for their additional effort. • When Management makes EVMS related decisions without feed back from program.

45. Know Your Weaknesses and Prepare • Use the last IBR results to your advantage. • Be ready to explain the plan to complete any open IBR issues. • Fix any known issues. • Have a “Program” answer for all known weaknesses. • Just because your CAMs did not get written up does not mean that they are compliant. • If possible have a independent review of the EVMS prior to the Validation.

46. Customer Interface • Ask your customer for clarification when in doubt. • Make your customer part of the solution not part of the problem. • Be very detailed when asking for guidance. • Ask for the appropriate citations when the results are to be flowed down to subcontractors.

47. Earned Value and the Control Account Manager (CAM) George Stubbs, CBT Workshop

48. Earned Value and the Control Account Manager • Why Be a CAM? • What is a Control Account Manager (CAM)? • The Control Account (CA). • What are a CAMs Responsibilities? • CAMs Activities and the Control Account. • The CAM and Control Account Performance. • CAM’s Monthly Requirements. • CAM Software Tools. • The CAM Notebook. • CAM Training. • What Makes a CAM Successful. • How to Undermine the CAM

49. Why Be a CAM? • First step into Management. • Opportunity to learn all aspects of project management. • Responsible for technical, schedule and cost performance. • Part of a progression of increasing responsibility. • CAMs become Program/Business Managers. • Program/Business Managers become Executives.

50. What is a Control Account Manager (CAM)? • The CAM plays a critical role in an Earned Value Management System. • Responsible for the planning, coordination and achievement of all work within a Control Account. • Provides a single authority for all scope, technical and cost issues for the Control Account. • Accountable to the Program Manager for Control Account Performance. • Authorized by and accountable to the PM for resources to complete cost, schedule and technical objectives. • Participates in Program Reviews, Integrated Baseline Reviews (IBRs), Validations and other project or management reviews. • Preparing for these reviews is a critical piece of the CAM’s role.