Iterative Project Management

1. Iterative Project Management Chapter 2 – How Do Iterative Projects Function? Iterative Project Management / 01 - Iterative and Incremental Development

2. Basic Philosophy • No matter the methodology (and there are many), the most important feature: methodology is iterative and incremental. • Regardless whether use-cases, pair-programming, scrum-meetings, feature-driven development, design by test approach or others is used, an iterativeapproach will greatly assist in producing predictable results. • Iterative development is characterized by small mini-projects (iterations) designed with a clear set of objectives producing a measurable executable (product) objectively assessed that incrementally advances a product of increasing business value. Lots of keywords in this. • The objective of this approach is simply to maximize chances for project success. Iterative Project Management / 01 - Iterative and Incremental Development 2

3. Mindset of Project Success • “Chaos” approach claims successful projects are finished on time, within budget, with all the features / functions present. • BUT: Using these criteria, most projects still fail; • with 16-24% succeeding; • And 15-40% ‘challenged’, • and 33-53% failed! • Successful: completed on time within budget; contains intended features/functions. • Challenged: project completed and operational but is over budget/time and has fewer features than originally intended • Failed: project is cancelled before completion. • Author claims that a successful project oftentimes facilitates organizationalchange, which changes ‘success’ criteria. • While the ‘above’ criteria are important, the real success is determined by the clear benefit to the business as measured and verified by business sponsors! Iterative Project Management / 01 - Iterative and Incremental Development 3

4. Mindset of Project Success • So, we are saying that sometimes providing “specific business functionality” may not be as important as delivering business benefit. These are not necessarily the same! • So what is ‘business benefit?’ • While we are certainly interested in projects that deliver business value in terms of functionality delivered, on time, and within budget, ‘businessbenefit’ itself may differ markedly from project to project and from stakeholdertostakeholder! • “Ultimately a project should be judged upon the value that it delivers to the business that commissions it, the customers that purchase its products, and the users that use them.” So: • It might be the application really had to be ‘first to market’ of its kind. • Might be a really innovative set of functionality; something really new! • Might be significant additional quality or performance… Iterative Project Management / 01 - Iterative and Incremental Development 4

5. Mindset of Project Success • Thus a project manager must understand the desired outcomes of the project are and what business results the product is expected to deliver. • Too often, development teams focus on technicalaspects of the project only to find themselves divorced from the “sometimes not too obvious” real business benefit desired by other stakeholders. Iterative Project Management / 01 - Iterative and Incremental Development 5

6. Success and the Iterative Project - 3 • In iterative development, the project may be adapted to changingrequirements as changingunderstanding of what constitutes success as the project progresses. • Poor results may occur if the views of some stakeholders, say the project managers or customers, is divergent from the developers. • An iterative approach helps us avoid this possibility of a project viewed as a failure by some yet a success by others. • We need to measure project success by focusing on desired businesssuccess and not necessarily blind adherence to some original plan that might seemingly focus on pure functionality. Iterative Project Management / 01 - Iterative and Incremental Development 6

7. Success and the Iterative Project • In this approach, each iteration has objectivesset by the management team collaborating with the development team (esp on technical objectives) and the customer team (esp where business/requirements – related objectives are defined). •  Remember, it is far more important (especially to management / customer) to deliver / test scenario(s), and/or set of implementedrequirementsorchanges resulting in a newrelease with more functionality than to produce a complete set of analysis and design documents for these features! Iterative Project Management / 01 - Iterative and Incremental Development 7

8. Evidence of Success • Pitfalls: (you may use these to fool yourself) • Use the iterative nature to excuse never finishing anything! • Allow results of one iteration to subvert results of previous iterations. A No-No! • Successful project management continuously monitors iterations so that an iteration • Takes a measurable step closer to desired result • Builds upon successes of previous iterations • Reducesprojectrisk • The success of a project is measured and becomes increasingly more evident iteration by iteration! Iterative Project Management / 01 - Iterative and Incremental Development 8

9. Key Characteristics of a Successful Iterative Project • Demonstrable, objectively measured progress • Incrementally increasing functionality • Continually improving quality • Continual risk reduction • Increasingly accurate estimates • Reducing levels of change • Convergence on a accurate business solution • Let’s look at some of these… On time, on budget, meeting the customer’s real needs. Iterative Project Management / 01 - Iterative and Incremental Development 9

10. Demonstrable, objectively-measured progress - 2 • Lots of ways: From book: • Number of products / documents produced • Number of LOC produced • Number of activities completed • Amount of budget consumed • Amount of schedule consumed • Number of requirements verified to have been implemented correctly. • Best one by far: number of requirements verified to have been implemented correctly • Others are indirect and may not measure real progress. • But we must test the release and we must record the amount of verified software! • Nice graphs in book re: requirements verified vs project schedule Iterative Project Management / 01 - Iterative and Incremental Development 10

11. Demonstrable, Objectively Measured Progress 100% Requirements Verified 0% Project Schedule / Iterations Iterative Project Management / 01 - Iterative and Incremental Development 11

12. Avoid Feature Creep - 1 • Feature Creep is natural and practically unavoidable. • Frankly, if the number of requirements grows, the team will not make schedule. • Solution 1: Just say no. • Will keep you on schedule, but will create ill-will and will likely result in escalation. • Often unforeseen features are ‘essential’ • Solution 2: Prioritize them and negotiate with Customer • What can be removed from a features list if the delivery date and resources remains constant • Be careful of ‘added resources!!!!!’ • Best to discuss this at iteration end when plans for the next iteration are being firmed up. • Underlying theme: have frequent deliverables and value present to the business sooner rather than later. • Do not recommend extending the current project. • Have more critical / core functions in earlier iterations Iterative Project Management / 01 - Iterative and Incremental Development 12

13. Incrementally Increasing Functionality - 3 Essential that each iteration produce more verified, demonstratable functionality without compromising sacrificing quality established in previous iterations. But there are issues here. This is not always the case: How so? Amount of Effort may have been less during an iteration (time off; illness…) Team productivity: As we progress through the iterations, we typically increase the productivity as the architecture becomes more stable, team members become more confident in the process, risk is reduced, as well as reduced breakage (ahead) Stability of earlier iterations: Inevitable for rework and breakage. Thus in a given iteration, the code growth for the ‘next’ iteration may come up short than that previously planned.

14. Incrementally Increasing Functionality The S curve depicting increased functionality shows that early iterations tend to yield ‘lesser’ increments Due to cost / effort of start up; familiar with environment; stability, … Lesser functionality too occurs at the end due to transitioning to the user community, training, customer service, latent bugs, clean-up, etc. Hence, middle iterations normally produce the most significant increases in high-quality increments that supplement previous increments. Too, risk is reduced, environment becomes more stable as well as the architecture, etc.

15. Incrementally Increasing Functionality Increment n + 2 What is this decrease? Increment n + 1 Increment n Iteration n Iteration n + 1 Iteration n + 2 Each iteration has more complete functionality than the one before. Iterative Project Management / 01 - Iterative and Incremental Development

16. Continually Improving Quality - 3 Healthy projects constantly assess quality and this must increase every iteration. But quality can be impacted by assessment at end of iterations: 1. testing does not totally address coded functionality (need more testing / development) and 2. just plain breakage (code does not pass tests). This regression is shown in the next slide.

17. Continually Improving Quality Regression in functionality from previous iteration(s). But overall higher quality code will be produced, as shown in the graph. Iteration n + 3 Iteration n Iteration n + 1 Iteration n + 2 Each iteration has less breakage than the one before. Iterative Project Management / 01 - Iterative and Incremental Development

18. Continually Improving Quality One problem with the effort to continuously improve quality is the perception that taking care of problems should be given precedence over adding new functionality But by ignoring / ‘pressing on’ in the face of defects causes more significant problems. The graph (see figure 2-8 in your text – page 61) shows that by subordinating the addressing of breakage, etc. causes more time spent in fixing ‘other’ problems and less overall progress and quality. And, this rate of ‘lower’ progress/quality increases… Too much emphasis on adding functionality leads to degrading quality… We know that earlier iterations must concentrate on stabilizing the architecture and reducing risk at the cost of increased functionality Thus care must be exercises in planning the iterations to address these parts of the project where architecture concerns and risk are high. Functionality can be added secondarily to these factors. Once risk becomes reduced and the architecture stabilizes, then more functionality will be added and a higher quality product continues its evolution. Avoiding accumulating defects will increase the quality of the increments.

19. Continual Risk Reduction – 2 We know that healthy projects address risk up front, as this reduces the likelihood of project failure. This is old news and is essential to early iteration planning. See next slide for graph: This graph reflects an immediate increase in risk up front rising to a high point, and then dropping off to become much lesser in importance as the project evolves. Earliest iterations are usually the most difficult as those items of high risk must be addressed. Too, a team rarely really understands all the risks up front. Thus there is a period of (book) risk exploration. What do we mean by risk mitigation? We divide projects into iterations to gain greater control over the project and mitigate risk.

20. Continual Risk Reduction High Controlled Risk Management Total Risk Exposure Risk Exploration and Resolution Low Project Schedule / Iterations Iterative Project Management / 01 - Iterative and Incremental Development

21. Controlling Change - 2 Again, there will be change and rework. But it is a matter of controlling and managing these activities! Change and rework generally come at a much higher expense later on in the project because the architecture is stabilized and so much functionality has been verified and integrated into the project. Bringing Change and Rework under control dramatically impacts overall project completion.

22. Controlling Change - 2 Early in lifecycle, we expect change – typically between 35% and 100% - as we become more stable and learn more about the project. Rework will then typically decrease and should drop to something below 25%. We must watch the stability of the interfaces – subsystems, packages, layers, etc. Major responsibilities of components! Earlier in project – no problem. Typically if these require change well into the project, then we likely have deeper problems. Watch for these!

23. Increasingly Accurate Estimates Accurate estimates for both short-term and long-term activities must be predictable. All estimates have an element of probability in them. Traditional estimates (COCOMO model) convered (as expected) as we approach end of project. Iterative projects do better due to revised estimates based on real progress measures / verified each iteration. We constantly revised our estimates and hence converge more quickly to actual costs As time progresses, the closer we get to actual results Margin of error decreases as time moves on in the iterative approach.

24. Increasingly Accurate Estimates Traditional approach for estimation. Source: COCOMO 2 Model Manual Iterative Project Management / 01 - Iterative and Incremental Development

25. Increasingly Accurate Estimates Iterative estimating: note quicker convergence and reduced margin of error. (Convince management of this) 4X Expected size rangefor a traditional project Expected size range for an iterative project 2X Relative Size Range X 0.5X 0.25X Project Schedule / Iterations Iterative Project Management / 01 - Iterative and Incremental Development

26. Estimating Facts • Facts are that • Estimating is so very poorly done. • Often estimates are dramatically influenced by management; perhaps negotiated. Why? •  “The reality is that reductions in schedule without corresponding reductions in scope have the effect of setting the project up for failure from the start.” p. 69

27. Estimating Facts - 2 • Facts are that • Software professionals don’t develop estimating expertise • We tend to be overly optimistic • Development teams don’t cope well with political problems. • There’s very little historical information to base estimates upon • Teams do NOT continuously revise estimates • But by continuously estimating via learning more, developing additional business value, assessing, and verification, and, equivalently, developing our own history, our estimates can become much more authoritative and result in more predictable outcomes.

28. Increasingly Accurate Estimates Note the margin of error decrease over time. X X X X X X Estimate of work to complete X X X Original Target Date Project Schedule Iterative Project Management / 01 - Iterative and Incremental Development

29. Reducing Levels of Change 100% Reducing levels of change? How so? Discuss. Rework (% of Total) 50% 0% Project Schedule / Iterations Iterative Project Management / 01 - Iterative and Incremental Development

30. High Defect Density /Defects per Line of Verified Code Low Project Schedule / Iterations Defect Density not only defect density, but their severity. If so, how so? Iterative Project Management / 01 - Iterative and Incremental Development

31. Convergence on an Accurate Business Solution The following perspectives converge iteration by iteration: • Discuss • What the customers think they need • What the customers expect to get • What the developers think the customers need • What the developers expect to deliver • What the users actually need • What the developers are actually going to deliver • What the customers actually going to get Iterative Project Management / 01 - Iterative and Incremental Development

32. More on Convergence… Reality is that users and business analysts often don’t know what they really want until they see it. Too close to the action in many cases… Too busy; bad attitudes; resistance to change; seniors vs newbees; turf;…. Often specifiers ‘need’ the world… until they see the cost and impacts on schedule. Nice thing is that early iterations address risk and force early problems to be resolved via demonstrations, proofs of concept, prototyping, etc….before long term project commitments need to be forthcoming.

33. Discussion: Resources • What skills do you need to iterate? • What are the key roles in an iterative project? Iterative Project Management / 01 - Iterative and Incremental Development

34. Project Management Requirements Management Development Architecture Assessment Team Responsibilities Who does what? What perspective do they come from? What specific skills do you see absolutely necessary? To iterate the key management and development skills need to be in place. Iterative Project Management / 01 - Iterative and Incremental Development

35. Increasing Enthusiasm, Morale, Collaboration, And Effective Teamwork Iterative approach is characterized by Regular demonstrations Assessments Retrospectives Feedback loops That reinforced team building and process improvement. BUT: team attitude is critical…

36. Team Attitude - 1… Oftentimes an iterative approach is taken due to previous project failures. Often many individuals / groups doubt the other’s ability to meaningfully contribute or have commitment ‘they’ have. DIFFERENCES: (for management…) (book) Iterative approach provides greater ability to see what’s happening Force issues to be dealt with immediately and not put off… Feedback is folded into the planning of iterations Actions taken to resolve issues. Iterative projects produce code almost immediately!! And these are addressed each iteration!! (unlike traditional aproaches)

37. Team Attitude - 2… Developers (paraphrased) commonly doubt the customer’s commitment to the project and question their willingness to become actively engaged in steering and assessing the project through the iterative elaboration of requirements providing feedback on the iteration demonstrations and contributing to the iteration assessments. Customers often question the development team’s attitude toward their (customer’s) taking a more central role in the projects and actually listening to their inputs.

38. Summary and Review • Maniacal focus on producing working software • Something ‘runnable’ produced every iteration • Objective measurement of progress • Continuous integration and testing • Active reduction of risk • Incremental completion …. • Convergence Iterative Project Management / 01 - Iterative and Incremental Development