UKSMA 2005 Project Estimating – Theory and Practice – Beyond Talking Paul Goodman

1. UKSMA 2005 Project Estimating – Theory and Practice – Beyond Talking Paul Goodman

2. Our Starting Point • “Our customers quite rightly complain. • It is not the fact that we get it (our estimates) wrong! • It is that we are always late, never early!” • CIO’s recent comment on the state of estimating and the effect on customer satisfaction

3. Topics for Discussion • Why is estimating accurately important? • Where should we start our improvement from? • A Case Study Overview

4. The Importance of Accurate Estimating Basic principles: • Estimation accuracy is a critical success factor for most businesses • Getting it wrong can adversely affect your company bottom line And • Will certainly upset your customers

5. The Importance of Accurate Estimating • We should not expect accurate estimating to be “natural” i.e. to just happen But • It is possible to estimate, after requirements specification, to within 5% of actuals Accepting 200%+ over runs is not the way it has to be

6. Starting Improvement • Appreciate that this will not happen overnight • It will probably take at least one year elapsed to effect broad sustainable improvement • Get to grips with some basic principles • Some are facts of life but they still need to be clearly recognised and stated • Others need to be embodied within your estimating processes

7. Facts of Estimating Life • Estimation inaccuracy is the difference between the estimate and the actual • If reducing this difference is important to you, make it important to everyone with estimating responsibility • If it is not important to you – STOP NOW! • Estimates and actuals need to be recorded in order to enable and to show improvement • Appreciate that people are genuinely and almost universally optimistic • Optimism is dangerous when estimating!

8. Estimating Principles • The estimator has the last word • Estimates are estimates - use bounds and a most likely • The estimate is dynamic - re-estimate • Use a sanity check, more than one technique • Keep information

9. Project Completion Actuals Final Size Project Completion Recording Information Effort Initial Estimation Re-estimation First Estimate Revised Estimates Metrics Database Initial Size Revised Size Change Control Size Identify the WEDGE factor

10. The Wedge Factor final requirements base line requirements 70% growth!

11. The Need for Sizing Requirements Effective sizing at an early stage is a pre-requisite to accurate estimation Common Sizing Techniques: • Mark II Function Point Analysis • IFPUG Function Point Analysis • Work Breakdown Structure/Task Sizing • Line of Code Sizing for Infrastructure Projects

12. Using Estimation Tools • There are many estimating tools available on the market • They can be very useful • They do need to be used with care • Be aware of the need to calibrate

13. Estimating Case Study

14. Agenda • Introduction and Process • Sizing • Costing • Effort and timescales • Summary

15. Introduction • Meta Group UK was requested by XXX to provide size estimates for the YYY System (a GIS application) • META Group were further asked to provide indicative and comparative costs estimates based on industry benchmarks

16. Conduct of Assignment • Chosen size metric was Mark II Function Point Index (FPI) • This was to be estimated based on best available information provided as documentation by XXX • Clarification of issues on documentation was provided by XXX • Documentation was reasonable and facilitated the use of a number of FPA estimation approaches • (Fact of life – the more limited your information, the less able you are to estimate)

17. Lifecycle Coverage • Our estimates will include the following elements of a project: • Requirements • Design • Build • Integrate • Test • Project Management • Quality Assurance • Excludes effort associated with: • Architecture and infrastructure (plan, build or run) • Business input

18. Cost/Effort estimate • Function Points are an estimate of size, not cost • The cost of delivering a system will depend on many factors including: • Maturity of the development teams • Stability of requirements • Tools and processes used • Quality of code developed (speed of test/acceptance)

19. Findings • Size estimation using three techniques: • Structured Expert Judgement; (Confidence is +/- 40% but tends to underscore) • Data Model Approach; (Confidence is +/- 25%) • Functional Decomposition (calibrated through Data Model Approach); (Confidence is +/- 20%) • (These are in the public domain – see “Sizing and Estimating Software in Practice”, Treble and Douglas)

20. Size Estimation Results META recommendation is that most likely size estimate be taken as 3,500 FPts

21. Sizing by Contribution • Contribution to total size made by Enquiry/Report functionality appears low • 18% of total business functions are Enquiry/Report type contributing 12% of total functionality • This may be perfectly valid but raises a flag – “Is XXX confident that reporting functionality has been adequately identified?” • It is stressed that this observation is based only on estimated counts.

22. Cost Estimate 1 • Assumptions: • System size is 3500 Function Points • Average lifecycle delivery rate of 15 FP / Man Month (mid-point performance within META Group’s AD benchmark) • Note: GIS type systems tend to achieve lower productivity levels than some other system classes • Blended ‘daily rate’ of £650 per day (mid-point in META Group’s Labour Rate benchmark) Note: Based on 22 working days per month

23. Cost Estimate 2 • Assumptions: • System size is 3500 Function Points • Based on International Software Benchmark Standards Group median productivity of 0.14 FP per Person Hour • Blended ‘daily rate’ of £650 per day (mid-point in META Group’s Labour Rate benchmark • This is an ‘optimistic’ estimate as this benchmark tends to reflects ‘best in class’ development productivity Note: Based on 22 working days per month and 6 effective hours per day

24. Cost Estimate 3 • Assumptions: • System size is 3500 Function Points • Based on CCTA median productivity of 0.07 FP per Person Hour • Blended ‘daily rate’ of £650 per day (mid-point in META Group’s Labour Rate benchmark • This is an a more pessimistic estimate based on a less mature development environment Note: Based on 22 working days per month and 6 effective hours per day

25. Costing Summary • META Group propose that the yyy application, based on documentation provided, is approximately 3500 FP in size ( with a likely accuracy of +/- 20%) • All evidence suggests that the cost range is between £2.7m and £5.4m based on market blended rate of £650 per day • META Group have no appreciation of the development maturity of the potential suppliers but do note that this has a major impact on costs • Cost may be reduced by the use of pre-packaged software components and the effective use of code generation tools (if applicable) • Overall, our experience indicates that based on our labour rate the total cost is likely to be between £3.2m and £4.5m

26. Reality Check • At this point we were told that the latest in house estimate was in the area of £3M • Given the discussion about reporting functionality that was now considered “low” • So what about the duration estimates?

27. Delivery Timing • We looked at duration estimates based on the estimated size and the profile of the organisation/project (mainly using COCOMO 2000) • The indications were that delivery within a 12 month period (including test) is an extremely aggressive timescale • This is putting it politely • This was in line with the “gut feel” of the teams on supplier and client sides (but now they felt able to talk about it)

28. Summary • Our best estimate is that the application should cost between £3.2 and £4.5m based on market rates • Delivery in a 12 month period would require a team of 15-23 FTE (and considerable “good fortune”) • The supplier would need to focus strongly on its development performance to achieve these targets • We recommend the supplier use function point techniques through the process to assure XXX of best in class development performance and to control change

29. Conclusions • This exercise took approximately two weeks elapsed time • The approaches described and data used are mostly in the public domain (or available at relatively low cost) • The analysis provided an objective vehicle for discussion • Estimates were revised towards the upper bounds and these revisions were informed by the analysis • Duration estimates were relaxed somewhat • Nobody discounts the need to apply some judgement • The project is on track to deliver within budget