940 likes | 1.22k Views
Cost Estimation. Cost Estimation. “The most unsuccessful three years in the education of cost estimators appears to be fifth-grade arithmetic. Norman R. Augustine. Goal.
E N D
Cost Estimation • “The most unsuccessful three years in the education of cost estimators appears to be fifth-grade arithmetic. • Norman R. Augustine
Goal • The cost estimation community is working to improve estimations so that sophisticated organizations can produce products with 5% of the estimated cost (instead of 10%).
Goal • The cost estimation community is working to improve estimations so that sophisticated organizations can produce products with 5% of the estimated cost (instead of 10%). • The typical software organization struggles to avoid estimates that are incorrect by 100%.
Goal What does “incorrect by 100%” mean? • The cost estimation community is working to improve estimations so that sophisticated organizations can produce products with 5% of the estimated cost (instead of 10%). • The typical software organization struggles to avoid estimates that are incorrect by 100%.
Requirements for Estimations • What are cost estimates used for?
Requirements for Estimations • What are cost estimates used for? • What are the important characteristics of a cost estimate?
Requirements for Estimation • Timely • It is of limited value to provide the estimate after the project is complete • How early?
Requirements for Estimation • Timely • It is of limited value to provide the estimate after the project is complete • How early? • Before implementation • Before complete design • Maybe before complete requirements
Requirements for Estimation • Timely • It is of limited value to provide the estimate after the project is complete • Accurate • How accurate is enough?
Requirements for Estimation • Timely • It is of limited value to provide the estimate after the project is complete • Accurate • How accurate is enough? • Enough for planning, bidding, scheduling
Scenario • A typical software development shop. A boss is speaking with the lead developer. • (I need two volunteers to play these parts.)
Scenario • What’s up with the boss?
Scenario • What’s up with the boss? • What does the boss think about the employee?
Problem with Scenario 1 • The boss wasn’t asking for an estimate. The boss was asking for a plan to hit the target. • Your boss may or may not know the difference.
Scenario 2 • What’s the difference?
Definitions • Estimate • Target • Commitment • Plan
Estimate • 1. A tentative evaluation or rough calculation. • 2. A preliminary calculation of the cost of a project. • 3. A judgment base upon one’s impressions; opinion • American Heritage Dictionary, 2nd Ed. 1985
Targets • “We need to have the prototype ready by the end of the semester.” • “These functions should be implemented by August 30 when the contract ends.” • “The cost of the project is limited to $250,000, because that’s the maximum budget.” • “We have to ship 7.0 by second quarter next year, because I have a reunion to attend in July.”
Commitment • Target: description of a desirable business objective • Commitment: a promise to deliver defined functionality at a specific level of quality by a certain date • Plan: a sequence of steps to achieve a goal (may include a schedule)
Terms • Note that target, estimate, and commitment are not the same concept, and the dates given for these may differ.
Scenario 3 • Suppose I give an estimate of 90 days. • What does this mean?
Scenario 3 This says there is a 100% probability of delivering on this schedule. In order for the number to have value, we need to know what the variance is. How likely are we to hit this estimate? (Usually, this is a target, not an estimate.) Suppose I give an estimate of 90 days. What does this mean?
Scenario 3: Bell curve • What does it mean? • Is this more accurate?
Scenario 3: Bell curve What assumptions does this make? • What does it mean? • Is this more accurate?
Scenario 3: Realistic • What does this mean? • Why is it shaped that way?
Scores • How many with 10 correct? • 9? • 8? • 7? ….
Math of the expected distribution • If we have a 90% probability for any single answer, then: • Probability of getting all 10 correct: .9^10 = 34.9% • Probability of getting 9 correct: (.9^9*.1)*10 = 38.7% 8: .9^8*.1^2*45 = 19.4% • For any given combination, .9^8*.1^2. But there are 45 different ways to put two wrong in a list of 10. • You can put the first wrong answer in any one of the first 9 places. • If you put it in the first spot, there are 9 places to put the second. • If you put it in the second spot, there are 8 places for the second. • And so on. • 9 + 8 + 7 … + 1 • 1 + 2 + 3 … + 9 • -------------------- • 9*10 = 2x, x = 45.
Math of the expected distribution Conclusion: with a 90% confidence, you have a 93% chance of getting 8 or more correct. • If we have a 90% probability for any single answer, then: • Probability of getting all 10 correct: .9^10 = 34.9% • Probability of getting 9 correct: (.9^9*.1)*10 = 38.7% 8: .9^8*.1^2*45 = 19.4% • For any given combination, .9^8*.1^2. But there are 45 different ways to put two wrong in a list of 10. • You can put the first wrong answer in any one of the first 9 places. • If you put it in the first spot, there are 9 places to put the second. • If you put it in the second spot, there are 8 places for the second. • And so on. • 9 + 8 + 7 … + 1 • 1 + 2 + 3 … + 9 • -------------------- • 9*10 = 2x, x = 45.
What we expect at 90% confidence Historical data Us last year
Questions: • Did you feel pressure to make your ranges wider? Or narrower? (Why?)
Questions: Did you feel pressure to make your ranges wider? Or narrower? (Why?) Where did the pressure come from?
Questions: Did you feel pressure to make your ranges wider? Or narrower? (Why?) Where did the pressure come from? Is estimating the volume of the Great Lakes anything like estimating software?
Questions: Did you feel pressure to make your ranges wider? Or narrower? (Why?) Where did the pressure come from? Is estimating the volume of the Great Lakes anything like estimating the impact of new programming tools on productivity, the productivity of an unidentified person, or the cost of developing software with no specification?
Accuracy and the cost of inaccuracy • What is the cost of overestimating?
Accuracy and the cost of inaccuracy • What is the cost of overestimating? • Parkinson’s Law: work expands to fill the time available • Goldratt’s Syndrome: People procrastinate until the last moment to start
Accuracy and the cost of inaccuracy What is the cost of underestimating?
Accuracy and the cost of inaccuracy • What is the cost of underestimating? • Reduced effectiveness of project plans • Reduced chance of on-time completion • Poor technical approaches: Not enough time in requirements and design • Destructive late project dynamics • More status meetings • Interim releases • Fixing problems from workarounds
Recall cost estimation: • Sophisticated organizations: within 10% • Typical software organization: >100% • Estimates need to be timely and accurate • Estimate, Target, Commitment, Plan • Costs associated with overestimates • Costs associated with underestimates
How are we doing? • KSLOC is 1,000 lines of source code • MSLOC is 1,000,000 lines of source code • With your partner, what does this graph say?
Benefits of Accurate Estimates • Improved status visibility • Higher quality • Better coordination with non-software functions • Better budgeting • Increased credibility for team • Early risk information
Benefits of Accurate Estimates • Improved status visibility • Track progress by comparing actual to planned • Ability to make a plan • Higher quality • Better coordination with non-software functions • Better budgeting • Increased credibility for team • Early risk information
Benefits of Accurate Estimates • Improved status visibility • Higher quality • Less stress on developers • Schedule pressure can increase defect rate by 400% (Jones 1994) • Better coordination with non-software functions • Better budgeting • Increased credibility for team • Early risk information
Benefits of Accurate Estimates • Improved status visibility • Higher quality • Better coordination with non-software functions • Testing, documentation, marketing, training, support • Better estimation: tighter coordination • Better budgeting • Increased credibility for team • Early risk information
Benefits of Accurate Estimates • Improved status visibility • Higher quality • Better coordination with non-software functions • Better budgeting • obvious • Increased credibility for team • Early risk information
Benefits of Accurate Estimates • Improved status visibility • Higher quality • Better coordination with non-software functions • Better budgeting • Increased credibility for team • Not unusual for • team to estimate, • others (manages, marketers, sales staff) turn it into optimistic business target • Developers overrun • Others blame team • Early risk information