CSC444 Software Engineering

1. CSC444Software Engineering Release Planning Lecture 7

2. Capability Maturity Model • Developed at Carnegie-Melon Software Engineering Institute by Watts Humphrey. • Classifies an organization’s process maturity into 5 levels. • Each level is a group of practices. • The CMM is a roadmap for process improvement. • Should have substantially all practices in place for a lower level before proceeding to the next • Can be certified to a certain CMM level • Some similarities to • Malcolm Baldrige • ISO 9000 • Not universally agreed to be a good thing • But everyone agrees to pretend Lecture 7

3. CMM Levels Lecture 7

4. Lecture 7

5. Relationship to ISO9000 • ISO 9000 • Set of quality standards • Subset relate to software development • In essence • Must document the process • Must maintain “quality records” • These are auditable to ensure the process is being carried out • The process can be anything Lecture 7

6. Relationship to Top10 • Practices necessary to achieve CMM Level 2 (Repeatable). • With enough Level 3 (defined) added to attain ISO9000. • With some Level 4 (quantitatively managed) sprinkled in where most effective: • Defect arrival / departure rates • Estimates versus actuals • Metrics on process step completion • Defect attribution Lecture 7

7. Planning • Most important aspect of CMM Level 2 • Common flaws: • Make no plans • Make a plan, but don’t track it • Use Microsoft Project Lecture 7

8. Why Plan? • Not always a good thing • If no expected date • If no other expectations (e.g., expected functionality) • Planning can only slow you down • Required when • External pressures come to bear on release dates • Usually only happens a bit later in a software company’s business evolution • Not right at the start • Necessary for “crossing the chasm” Lecture 7

9. Crossing the Chasm, Geoffrey Moore (1991) Lecture 7

10. Gantt Charts Considered Harmful Lecture 7

11. Planning Essentials • What are we building? • By when will it be ready? • How many people do we have? Answer these and nothing more: not “who will be doing what?” nor “what are the detailed tasks required?” nor “in what order must the tasks be performed?” Lecture 7

12. Implementation Plans • Once planning is complete can then transform into a detailed plan • E.g., Microsoft Project • Detailed plan should not contradict the release plan • Not all of the project needs details beyond • Who do we assign it to • But some parts do • These plans may not be necessary • If no great inter-dependencies that can’t be worked out as you go • They hinder change as they are so cumbersome to change Lecture 7

13. Of Mice and Men • The essence of planning is uncertainty • Plans never “go according to plan” • Must embrace change, not close our eyes to it • Therefore: • Must track the plan always • Must react quickly to adverse situations • Must embrace changes in direction • Must re-plan quickly Lecture 7

14. Internal Changes • Estimation errors • Initial estimates contain a significant (one-sided) margin of error. • As plan progresses, variance in estimates lower • Developer-power leaving the project • Illness • Parental leave • Resignations • Budget cuts • Unexpected vacation plans • Unexpectedly low work hours • Unexpectedly low productivity Lecture 7

15. External Changes • New (big) customer desiring new functionality • Competitor release a product • Collaboration opportunities • Acquisitions and mergers • Sudden changes in customer needs • E.g., regulatory changes affecting them Lecture 7

16. The Difficult Question • What are we building? • May be hard for 1st release • Subsequent releases will have a big wish list • Pick the best looking ones, most demanded ones • Marketing product management decision • What will make us the most sales? • By when will it be ready? • Too soon: • Customers won’t be ready for a new release • Won’t want to install • Won’t want to learn • Won’t want to pay for it • Too late • Customers will forget about you • Competitors will pass you • Foregone revenues • How many developers? • Usually fixed for next release • Difficult question • Can we do all 3 at once? Lecture 7

17. A Common Happening • Often organizations will answer all three questions, but not address the difficult one. • Development management will want to please their masters, and will tend to agree to too much in a spirit of “gung-ho!” • Some managers firmly believe that over-commitment is the road to productivity. • “It’s a stretch, but we’ll pull it off” • Coders will say “it can’t be done” • but “that’s all they ever say”. • Massive sate of denial will set in. • Everybody will hope for a miracle • Nobody will accept any blame • Development management: we told you it would be a stretch • Coders: we said it could never be done • Marketing: you should have said something earlier • CEO: you all told me it was going fine • Yourdon’s “Death March”. Lecture 7

18. The Solution:Good Planning! • Does not need to happen this way. • But need courage and conviction. • Need common sense: • Is it even feasible to do what’s asked by the date required? • Don’t give an off-the-cuff answer • Even if it is obviously impossible • Put together a plan and demonstrate the facts. Lecture 7

19. Release Planning ReviewThe Product Lifecycle Lecture 7

20. ? Next Release Potential Requirements Eliciting Potential Requirements • Starts with a wish list • Stated as business requirements • Features or architectural enhancements Lecture 7

21. A Simple Release Plan Dates: Coding phase: Jul.1—Oct.1 Beta availability: Nov.1 General availability: Dec.1 Capacity: days available Fred 31 ecd Lorna 33 ecd … … Bill21 ecd total317 ecd Requirement: days required AR report 14 ecd Dialog re-design 22 ecd … … Thread support87 ecd total317 ecd Status: Capacity: 317 effective coder-days Requirement: 317 effective coder-days Delta: 0 effective coder days Lecture 7

22. Sizing the Available Resources • Who can work on the next release? • Must have skills and familiarity to contribute • For how long? • Must count workdays in the coding phase • Each resource available all that time? • Subtract estimated vacation • How dedicated to the next release • Work factor = w • Converts 8-hour (nominal, arbitrary) days to time available to code and unit test during the coding phase • E.g. w=0.6 means can dedicate 0.6x8 = 4.8 hours/workday • Accounts for • Other tasks, sickdays, meetings, weekends, ... • Range is 0 .. 9, usually 0.6 or so Lecture 7

23. Sizing Potential Requirements • Cost / Benefit analysis • Cost: financial + opportunity = person days • Sizing in ECDs • Inherent size of the work item • Who will work on it • The productivity of that person on that work item • Ensure units are well-understood (more later) Lecture 7

24. The Capacity Constraint • After all is done in a release Actual Resource Used == Sum of Actual Times for Features • This is always true. It is a constraint. • In planning, knowing that this must work out at the end, can estimate both sides and force the estimates to be equal Resource Estimate == Sum of Feature Estimates Lecture 7

25. p e r s o n s days 1 person-day A Geometric Analogy - Capacity Lecture 7

26. A Geometric Analogy - Requirement Lecture 7

27. A Geometric Analogy - Capacity Constraint It Must All Fit! Lecture 7

28. Release Planning • What to build F • By when to build it T F  N x T • Using how many people N • Need to build an initial plan that respects the capacity constraint • Need to continuously update the plan to maintain its adherence to the capacity constraint. Lecture 7

29. Most Common Problem • Comes from either • not knowing • knowing but hoping for the best (Yourdon Death March) (can happen initially or as we go) Lecture 7

30. Add time Cut features Both Dealing with Issues as they Arise Developer leaves the team Lecture 7

31. Other Happenings Lecture 7

32. Organizational Issues • Management must appreciate that software development carries with it certain inherent risks. • The business of a software organization is to manage and adapt as possibilities continuously become reality. • Ranting and raving is unproductive • With good data, good managers will make good decisions. Lecture 7

33. F = N x T N T The Quantitative Capacity Constraint • Post-Facto, the following relationship must hold. • But, it requires careful definition. We define carefully so that we know what it is we are trying to estimate, and how to compare actuals against estimates for post-mortem. Lecture 7

34. Number of Workdays: T • The number of full-equivalent working days for fork to dcut. • Subtracts • Weekends • Statutory holidays • “Company Days” • Subtracts anything we know in advance that nobody is expected to work. Lecture 7

35. Developer Power: N • The average number of dedicated developers per workday working during the T-day period. • Dedicated Developer? Lecture 7

36. Work Time & Dedicated Time • Work Time or Body Time • Defined as 8 hours per workday • Excludes weekends, stat. holidays, vacation entitlement. • E.g., 9-to-6 with 1 hour for lunch. • Dedicated Time • Uninterrupted hour equivalents. • Time dedicated to adding new features to the release. • Uninterrupted Time • 4 hrs with 30 min. of constant interruptions • Not 3.5 hrs of dedicated uninterrupted time – more like 2 • 2 hrs with NO interruptions at all Lecture 7

37. Examples of Dedicated Losses • Maintenance (tracking down and fixing defects) on previous releases • Other simultaneous projects • Team-leader duties (& helping others) • Meetings • Training • Unexpected, non-made-up days off (e.g., sick days) • Sales/marketing support • Loss of flow due to interruptions Lecture 7

38. Measuring N • Assume each developer understands the concept of a dedicated uninterrupted hour. • Get each of the n developers to record how many dedicated uninterrupted hours they spent in total during the coding phase. • hi is what’s in the time tracking system for the ithdeveloper. Lecture 7

39. Attributing N • diis the number of days available during the coding phase • vi is the number of vacation days they took during the coding phase • hiis as before Substitute to get back to: Lecture 7

40. Example • Bob called in sick for 2 days: accounted for in h • Bob took an afternoon off, but worked on the weekend to make up for it: accounted for in h Lecture 7

41. Features Features fk = dedicated hours / 8 it took to code the kth feature. Lecture 7

42. Post-Mortem • Imagine a time-tracking system that could track • hi,k,d • dedicated (uninterrupted) hours spent • by the ith developer • on the dth day • doing coding work on the kth feature • each such quantum would appear on both sides of F= N x T constraining them to be equal. • See section 5.10 in book for proof. Lecture 7

43. Example Release Plan • Sample Deterministic Release Plan Lecture 7

44. The Stochastic Capacity Constraint Lecture 7

45. Estimates • Estimates are never 100% certain • E.g, if we estimate a feature at 20 ECD’s • Not saying will be done in 20 ECDs • But then what are we saying? • Are we confident in it? • Is it optimistic? • Is it pessimistic? • A quantity whose value depends upon unknowns (or upon random chance) is called a stochastic variable • Release planning contains many such stochastic variables. Lecture 7

46. Confidence Intervals • Say we toss a fair coin 5000 times • We expect it to come up heads ½ the time – 2500 times or so • Exactly 2500? • Chance is only 1.1% • ≤ 2500? • Chance is 50% • If we repeat this experiment over and over again (tossing a coin 5000 times), on average ½ the time it will be more, ½ the time less. • ≤ 2530? • Chance is 80% • ≤ 2550? • Chance is 92% • These (50%, 80%, 92%) are called confidence intervals • With 80% confidence we can say that the number of heads will be less than 2530. Lecture 7

47. Stochastic Variables • Consider the work factor of a coder, w. • When estimating in advance, w is a stochastic variable. • Stochastic variables are described by statistical distributions • A statistical distribution will tell you: • For any range of w • The probability of w being within that range • Can be described completely with a probability density function. • X-axis: all possible values of the stochastic variable • Y-axis: numbers >= 0 • The probability that the stochastic variables lies between two values a and b is given by the area under the p.d.f. between a and b. Lecture 7

48. PDF for w • Probability that 0.5 < w < 0.7 = 66% • Looks to be fairly accurate. • Has a finite probability of being 0 • Has not much chance of being much greater than 1.2 or so • Drawing such a curve is the only real way of describing a stochastic variable mathematically. Lecture 7

49. Parameterized Distributions • “So, Bill, here’s a piece of paper, could you please draw me a p.d.f. for your work factor?” • Nobody knows the distribution to this level of accuracy • Very hard to work with mathematically • Usual method is to make an assumption about the overall shape of the curve, choosing from a few set shapes that are easy to work with mathematically. • Then ask Bill for a few parameters that we can use to fit the curve. • Because we are not so sure on our estimates anyways, the relative inaccuracy of choosing from one of a set of mathematically tractable p.d.f.’s is small compared to the other estimation errors. Lecture 7

50. e.g., a Normal for w • Assume work factors are adequately described by a bell-shaped Normal distribution. • 2 points are required to fit a Normal • E.g., average case and some reasonable “worst case”. • Average case: half the time less, half the time more = 0.6 • “Worst” case: 95% of the time w won’t be that bad (small) = 0.4 • Normal curves that fits is N(0.6,0.12). area = 68% Lecture 7