Project Monitoring and Control with EVA and Burn Graphs

1. Project Monitoring and Control with EVA and Burn Graphs Guy Davis Kendra Hamilton Ed Dantsiguer

2. Agenda • Origins of EVA • EVA Explained • EVA Examples • Shortcomings of EVA • Agile side of EVA • Burn Graphs • Tools • Discussion

3. Overview of EVA • “The essence of [Earned Value Management Systems] is that some level of detail appropriate for the degree of technical, schedule, and cost risk or uncertainty associated with the program, a target value ….is established.” Paul Solomon 2002.

4. History of EVA • PERT/Cost (1963) • DoD Cost/Schedule Control Systems Criteria (C/SCSC) (1967). • Government Performance and Results Act (1993), Federal Acquisition Streamlining Act, Title V (1994), Clinger-Cohen Act (1996) • EIA-748-1998 “Earned Value Management System” • Office of Management and Budget (Circular A-11, Part 7) (2003)

5. Motivation for EVA • Integrates work, cost, and schedule metrics. • Early warning signal. • Driving by looking in the front windshield instead of the rear view mirror. Statistical projection. • U.S. Government requirement.

6. Cost EarnedValue Schedule Technical Performance EVA Explained 1/3 • EVA concentrates on project management and control • Requires a number of tasks to be performed before utilizing EVA: • Work Breakdown Structure (WBS) • Creation of detailed plan (critical path plan) • Includes all activities that have to be performed, their durations, costs and relative contributions to the overall deliverable

7. EVA Explained 2/3

8. EVA Explained 3/3 • EVA is used to determine current project performance and estimate/forecast future project performance • Based on 3 data points: • Budgeted Cost of Work Performed (BCWP) • Actual Cost of Work Performed (ACWP) • Budgeted Cost of Work Scheduled (BCWS)

9. Budgeted Cost of Work Performed • Planned (budgeted/estimated) cost of work that has been completed until this point • Answers: “How much was performed work supposed to cost?” • Based on features/activities completed and the budgeted amount for these features/activities in the original project plan

10. Actual Cost of Work Performed • Actual cost of work that has been completed until this point • Answers: “What was the actual cost of work actually performed?” • Based on features/activities completed and cost of these features/activities in real life

11. Budgeted Cost of Work Scheduled • Planned (budgeted/estimated) cost of work that was supposed to be completed • Answers: “How much work should have been done and how much was it meant to cost?” • Based on features/activities planned/scheduled and the budgeted amount for these features/activities in the original project plan • Budget at Completion (BAC) is the total funds allocated (budgeted) for this project to complete

12. Derived Metrics • Schedule Variance (SV) • SV = BCWP – BCWS • Compares what is done with what was supposed to be done • SV < 0  project is behind schedule • Cost Variance (CV) • CV = BCWP – ACWP • Compares actual project cost with budgeted project costs • CV < 0  project is over budget

13. Schedule/Cost Performance Index • Schedule Performance Index (SPI) • SPI = BCWP/BCWS • SPI < 1  project is behind schedule • Cost Performance Index (CPI) • CPI = BCWP/ACWP • CPI < 1  project is over budget • Cost Schedule Index (CSI) • CSI = CPI * SPI • CSI < 1  project is not tracking to plan • The further away CSI is from 1, the less likely is successful project recovery

14. Using EVA Metrics in Project Control 1/2 • Each individual EVA metric is not greatly useful on its own • Metrics need to be considered as a group • Ex: Just because a project has a CSI of 1 does not imply that the project is doing well – it may be well ahead of schedule while also being well ahead of its budget

15. Using EVA Metrics in Project Control 2/2 • ACWP metric can be used to project future activity costs/durations • This is called Estimate To Completion (ETC) • The end of the projected ETC curve is the Estimate At Completion (EAC) • estimated schedule and cost required to complete the project based on current productivity and spending • Comparison between EAC and BAC shows is the project is likely to be on schedule and/or on budget • Variance at Completion (VAC)  schedule difference between BAC and EAC

16. Recommended Performance Metric Values Per the “U.S. Marine Corps Acquisition Procedures Handbook,” June 1997

17. EVA Task Types • Discrete Effort • Activities with start and end time that result in deliverables • Apportioned Effort • Effort required to support discrete effort tasks (Ex: inspections, quality control) • Proportional to the type/size of discrete effort tasks that they support • Level of Effort • Overhead activities with no concrete deliverables (Ex: management and administrative activities)

18. Crediting Earned Value • Discrete Effort • Credited upon completion with actual cost and duration tracked • Apportioned Effort • Credited upon completion of related discrete effort tasks • Level of Effort • Credited according to plan (regardless of actual cost and duration)

19. Crediting Earned Value Methods • Milestone Events • Weighted Milestone Gates • Percentage Complete • Fixed Formula • Level of Effort • Percentage Complete and Milestone Gates

20. EVA Example 1/2 • Planned/Scheduled Data: • Duration of 10 months • Includes 10 features with multiple tasks • Budget of $100 million • Actual After 6 Weeks: • 55% of the work has been completed • $85 million has been spent

21. EVA Example 2/2

22. Success Factors for EVA • Quality of the baseline; need to include all details. • Take action early based on performance indicators. “I hate everything that merely instructs me without augmenting or directly invigorating my activity” Goethe

23. Shortcomings of EVA • Based on past performance; assumes constant rate of spend and value creation • Assumes a direct relationship between time and cost. • Value measured in technical components, not expected business value. • Project must be fully defined at outset; the devil is in the details. • Time required for measuring project’s progress

24. EVA Tools • Excel • Welcom “Cobra” http://www.welcom.com/ • Schedulemaker http://www.schedulemaker.com • Planisware “OPX2” http://www.planisware.com/ • RiskTrak http://www.risktrak.com/index.htm • Winsight http://www.cs-solutions.com/ • Primavera Systems http://www.primavera.com/

25. EVA compared to Agile • Full project view vs. Iterative view • Tasks fully defined vs. Changing requirements • Attempts to forecast future vs. Determination of next iteration • EVA is not suitable for truly “Agile” projects

26. EVA in an Agile/Iterative Project • Approach 1: • Stories = BCWS • Tasks = BCWS in more detail • Assignments = ACWP • Velocity = BCWP • Testable requirements = 0% or 100% BCWP

27. EVA in an Agile/Iterative Project • Approach 2: • Do EVA on individual iterations • Approach 3: • Generate micro estimations for current iteration and macro estimations for future iterations

28. Burn Graphs • Origins of Burn Graphs • Cumulative flow diagrams from lean production • Goal: to provide a succinct view of progress. • Allows project sponsors to steer the project. • Allows scrum master report visually to stakeholders. • Allows the team to gain experience estimating by getting direct feedback. (Empowering)

29. Burn-Down Graphs • Shows remaining estimated effort on item • Usually in ideal engineering time (IET hours) • Calculated for any level of task abstraction

30. Scrum Backlog Graphs • Product Backlog Graph • High-level view of overall project progress. • Completion date: work left versus resources available. • Quantitative tool for making trade-offs. • Sprint Backlog Graph • Detailed view of a single sprint's progress. • Sprint signatures: compare current with past results. • First notice of schedule slips seen here.

31. Not always pretty...

32. Burn-up Graphs • Shows progress on completion of item • Usually displays percentage complete • Just the inverse of a burn-down graph

33. Cumulative Flow Diagrams • Tracks number of features (and status) over time • Better for reporting than: • % complete graph of feature milestone percentages • Features completed over time

34. Drawbacks Metric choice is key Over-simplification? Hides dependencies Should be able to zoom to see levels Traditionalists will resist implementation? Overview of Burn Graphs Benefits • Easy to compile/track • Feedback to the team and status to the customer • Highlights: • Schedule slips • Scope creep

35. Burn Graph Tools • Open-source • Outreach Project Tool (OPT) • XPlanner • Commercial • Version One • MS Excel

36. References • Anderson, David J. “Using Cumulative Flow Diagrams with FDD”. Feature Driven Development. 2003. http://www.featuredrivendevelopment.com/node/view/515 • Anderson, David J. Agile Management for Software Engineering. Prentice Hall. 2003 • Alleman, Glen B., Henderson, Michael, “Making Agile Development Work in a Government Contracting Environment” Proceedings of the Agile Development Conference, IEEE, 2003. • Fleming, Quentin W., Koppelman, Joel M., “Earned Value Project Management: A Powerful Tool for Software Projects”, Crosstalk, July 1998. • Hayes, Heather, “Using Earned-Value Analysis to Better Manage Projects”, Pharmaceutical Technology, February 2002. • Howes, Rodney “Improving the Performance of Earned Value Analysis as a Construction Project Management Tool”, Engineering, Construction and Architectural Management, 2000. • Schwaber, Ken and Mike Beedle. Agile Software Development with Scrum. Prentice Hall. Upper Saddle River, NJ. 2002 • Solomon, Paul J., “Practical Software Measurement, Performance-Based Earned Value,” Crosstalk, September, 2002.

37. Discussion Points • Can EVA be applied to agile projects? • What metrics would you use for burn graphs? • Would you adopt burn graphs at your organization?