Linda Esker Kathleen Dangle Fraunhofer Center Maryland

1. Developing An Early Project Cost Estimate:An Applied Case Study From Project Concept through Contract Award Linda Esker Kathleen Dangle Fraunhofer Center Maryland

2. Project Overview The Early Project Cost Estimate Estimate Phases Challenges Strategy Approach: Methodology and Models Used Decisions to go forward Summary / Lessons Learned Topics Covered

3. Project Overview • A large Ground Space Communication System • COTS-intensive • For both HW and SW • 5-year project duration • Many areas highly scientific • Need appropriate technical expertise as well as those who understand what is involved in the cost estimate, especially SW • Would require a blended estimation team • Development of the estimate began in the earliest initial concept phase

4. Cost Estimate Began Before Typical 1st Milestone Life-Cycle Phases RFP Reviews

5. The Early Project Cost Estimate • Cost estimate developed bythe projectto estimate government and contractorproject costs for budget/funding approval • Activity played a significant role in the development of the project • Value of the activity included not just the resulting estimate, but also drove understanding, evolution, and “selling” of the project concept • State of the practice for early project estimates: • Typically use general, high-level, “personal” heuristics/ rules of thumb by subject matter experts • Not able to effectively use estimation tools because details required as inputs are not known early in the project • Process followed not as disciplined or structured (or repeatable) as expected in estimates during later phases of project lifecycle

6. The Early Project Cost Estimate (cont.) • The good news: With government emphasis on better project estimates, projects now need a basis of estimate and confidence levels • The bad news: Solid estimates face challenges that make use of estimation tools an uphill battle • Politics — Organizational budgets have a life of their own, are highly competitive, and defy comprehension for those not in the political know • Personal opinion and aggressive (or naive) optimism, “This should be easy…” • Distrust of what they do not understand

7. Early Project Estimate Phases 1 2 3 4 • “In the Beginning…” • Forming the concept • “Let there be light…” • Maturing the project’s architecture and requirements • Defining a project lifecycle & WBS • Defining the estimation process and models • “And it was good…” • Iterating on the architecture, schedule, models, and estimate • Evaluating options and “what ifs” • “And it was better…” • Formalizing Estimate Confidence Lessons Learned Lessons Learned Lessons Learned

8. Early Project Estimate Phase 1 • Started with: • Top-down approach • Fully understanding of the scope • Understanding SW and HW requirements of the project • Examined use of a commercial estimation tool • Found it was difficult to use at this very early stage • Needed more details than what was available • Input was constantly changing as project investigated technologies and needs and tool data cumbersome to update • Management did not trust estimation tool results • Wanted greater insight into and control on how estimate was determined • Wanted costs broken down into their areas of interest

9. Early Project Estimate Phase 1 Decision: Next use bottoms-up approach for greater accuracy • Concept studies performed to develop notional architecture • Estimate used: • General parametric models • Expert judgment • COCOMO • Spreadsheets • HW: Developed MEL (Master Equip. List) • SW: Used analogies/LOC • Percentages used to estimate many areas: • Management • Contingency, reserve & inflation • Spread of labor, HW, SW by fiscal year • Other technical unknowns Approach • Phase Start: Blank page; immature • requirements; forming team; gathering • historical data; top-down approach Challenge: Create initial estimate with minimum information Strategy: Formulate concept • Lessons Learned • Initial informal “off-the-cuff” estimates can derail a project from the start • Do not count on Mgmt. acceptance of tool estimate results

10. Early Project Estimate Phase 2 • Enhanced the Estimate with some structure: a notional architecture, schedule, and bottoms up-analysis • Stabilized requirements to refine the HW and SW estimate • COCOMO schedule and effort distributions allowed us to distribute costs over the timeframe to allocate budget to fiscal years • COSYSMO could now be used to validate engineering SME estimate • Needed to use requirement expansion factors • Project still going through definition and needed to adjust for changes

11. Early Project Estimate Phase 2 Decisions: Pursue 2 independent paths (dev., deploy.); focus on things missing • Migrated approach from parametric overlay to bottoms up • Established physical notional architecture to organize costs • Approach helped drive engineer thought process • Estimate used: • Expert judgment • Analogous estimation - some historical data • COCOMO for SW effort & duration • % still used to estimate • Management (based historical data) • Contingency, reserve, inflation, & unknowns • Spread of labor, HW, SW by fiscal year Approach Strategy: Use bottoms- up approach; define models; HW/SW by arch • Phase Start: Immature requirements; notional architecture beginning to evolve • Lessons Learned • Hybrid of bottoms-up & parametric modeling works best • Everyone understood what was known (justifiable) and unknown Challenge: Establish sound basis for good estimate

12. Iterative Development of the Estimate Labor, Material, Cost Phasing • WBS • (By Notional Architecture) Project Lifecycle Schedule Estimation Model • RESOURCES • Hardware • Antenna • Antenna part 1 • Antenna part 2 • Computers • Server • Workstation • Software • Software item 1 • Software item 2 • Engineering tasks • Integration Tasks • Transition tasks Items, Costs SLOC COCOMOModel Effort, Duration Effort Notional System Architecture • Master Resource Sheet of • Hardware • Software • Engineering Effort • Integration Effort • Transition Effort

13. Early Project Estimate Phase 3 • Areas of Focus • Identifying a good system model on which to base confidence in the estimate—having a solid Basis of Estimate (BOE) • Ensuring Implementation schedule is realistic • Understanding changing expectations and being flexible to deal with change ►Filling voids in analogous systems by using SW SMEs to provide sizing information ► Using a notional system to obtain sufficient information for an early project estimate and trying to avoid over-engineering the perfect system • Key Accomplishments • Met goals of project: Early Project Estimate for RFP; presentation to HQ; understandable/supportable basis for budget – SW: – HW:

14. Early Project Estimate Phase 3 Decisions: Ready to go; freeze estimate for RFP • Incorporated inputs from Trade Studies and another independent cost estimate • Aligned cost structure, schedule, WBS, and notional architecture • PM worked with all technical teams to understand basis of estimate (BOE) • Loaded Resources by skill level & labor rates • Spread labor, HW, SW costs by fiscal year based on schedule • Only PM remained % • Enhanced estimate model to allow: • Many different views (architecture elements,high cost drivers) • Investigation of options ("what if" a ground station were eliminated?) Approach Strategy: Use 2 separate Teams (dev. & deploy.); merge results • Phase Start: Matured requirements & team; near complete notional architecture • Lessons Learned • BOE review sessions need a strong driver • The better the estimate fidelity the more useful for “holes” & “what ifs” • Working as a team creates buy-in and project team is much smarter Challenges: Honing the estimate;

15. Early Project Estimate Phase 4 Decisions: Project has confidence with expected range • Performed Risk Cost Analysis • Focused on high cost drivers • Optimistic • Most likely • Pessimistic estimate • Used triangular distribution and Monte Carlo to simulate confidence ranges • Compared dispersions with expected ranges • Detailed resource items enabled analysis of procurement long lead items & phasing Approach • Phase Start: RFP Released; Project estimate frozen; fewer distractions Strategy: Examine Estimate Cost Risks • Lessons Learned • A solid early Project cost estimate enables developing confidence levels and understanding where the estimate falls within the levels before contract start Challenge: Understand Confidence in Estimate

16. COCOMO Lessons Learned • At this phase, the summary COCOMO tables are very helpful. However, • They do not give consistent information • Percentages do not add up to 100% • Total effort has two different values • This makes them appear to be unreliable • Had to make adjustments

17. COCOMO Lessons Learned • Modular approach of COCOMO products is ideal • Project had definite need for a COCOTS-like tool, but • Seemed too immature • Decisions on COTS were not controlled by government • Overlap of COSYSMO and COCOMO II makes management wary of using them together • Need some better quantification of the overlap in the estimate

18. Summary / Lessons Learned • Start early: Plan for iterations of the estimate • The project estimate will iterate through phases and different estimation techniques as more information is learned • The estimation process can aid in the maturation of the project concept, requirements, and schedule • Be Adaptable/Flexible:Standard tools cannot handle all of management’s needs • Early on there is not a clear definition/agreement on how to proceed and implement the project; need to change and adapt • Be sure your cost estimation tools are flexible • Need to handle what is known about the project at early and also later phases of the evolution • COTS may not match all your needs—a hybrid approach works well

19. Summary/Lessons Learned • Be Mindful of Management: Need to work with high-level management and keep them in the loop • Early “off-the-cuff” promises can derail a thorough estimation process • Don’t promise too much too soon—early promises cannot be rescinded • It is imperative that the higher management teams have faith in the estimation work • You need to thoroughly understand (and justify) how a tool’s model handles all aspects of the estimate • Even if not accepted for an end result, use tools to also give a sanity check for any ad hoc estimation practices

20. Summary / Lessons Learned • WBS is Critical: Projects need a WBS that can help them collect data and learn • Lobby for better WBS to facilitate data and information collection when necessary • Current WBS templates/guidelines make it difficult to extract software from other parts of the system development work • Project was afraid to specify that WBS differentiate between software and other activities • Would love to see this addressed as a SW best practices

21. Questions For more information contact us at: Linda Esker lesker@fc-md.umd.edu Kathleen Dangle kdangle@fc-md.umd.edu