Chapter 2: Iterative, Evolutionary, and Agile

1. Chapter 2: Iterative, Evolutionary, and Agile

2. Choice of Software development Process • Iterative because the entire project will be composed of mini-projects and will iterate the same activities again and again (on different parts of the project) until completion. • Evolutionary (or incremental) because the software grows by increments (as opposed to the traditional Waterfall model of software development). • Agile because we will use a light approach to software development rather than a very rigid one (which may be needed for a safety-critical system for example) • This kind of approach seems better at treating software development as a problem solving activity; and the use of objects makes it amenable.

3. Unified Process • A relatively popular iterative software development process • Iterative and evolutionary development uses relatively early programming and testing of a partial system, in repeated cycles. • It typically also means that development starts before the exact software requirements have been specified in detail; • Feedback is used to clarify, correct and improve the evolving specification; • This is in complete contrast to what we usually mean by engineering! (Why? Why do you think it works better this way for software?)

4. What is the Unified Process (UP or RUP) and Why? • What: • By the authors of the UML: Jacobson, Booch and Rumbaugh in 1999 • Not part of the UML ! • Why: • We need a process; • It is popular for OO project; • It is a good middle-of-the-road process;

5. Unified Process (cont.) • The UP is very flexible and open and can include other practices from other methods (e,g,Extreme Programming (XP) or Scrum). • E.g. XP’s test-driven development, refactoring can fit within a UP project; So can Scrum’s daily meeting. • Being pragmatic in adapting a particular process to your needs is an important skill : all projects are different. • The OO skills and techniques that we will see are not specific to the UP: • They can be applied within any software development process; • The UP is just a framework for a project, it brings some structure to what we do.

6. What is Iterative and Evolutionary Development? • An iteration is a time-boxed (e.g. 3 weeks) mini-project during which an increment of the software is developed and integrated into the current project. • Each iteration is made up of the following sequential activities: requirements analysis, design, implementation and testing. • The software grows by increments, and is refined (evolves) as we go along via feedback: Iterative and Evolutionary Development. • Be careful! -- an increment is not a prototype that sometimes work and sometimes doesn’t; it should represent a properly analyzed, designed, coded and tested chunk of the future software.

7. Fig 2.1 Iterative and Evolutionary Development

8. Example of three-week agile iteration (not always suitable) • The last iteration is debriefed and suggestion for process improvement are noted; • Feedback from previous demonstrations and subsequent requests for change are integrated in this iteration; • All the artefacts (use cases, UML diagrams, code …) from the previous iteration are updated and synchronized with each other; • Meeting clarifying the goals of the next iteration; • Use cases may need to be refined; • OOA is performed by the team in pairs with UML using whiteboard and capturing outcomes using a digital camera; • OOD is performed by the team again probably by pairs. Previous UML artefacts are updated; • Most of the remaining days are spent on implementation and testing (with maybe re-design if necessary); • The last day should be spent tidying up and documenting; • (Other project management activities such as demonstrations, quality evaluation, etc. have not been detailed here)

9. Agile iteration (cont.) • Do not rush to code nor do a long drawn-out all-detailed design phase; there should be team-dependent compromises. • The result of each iteration is an executable but incomplete system: • It should not be a hack nor a prototype; • For this approach to work we must not have to go back and change previous design decisions or code (that’s a hack); • Whatever we do within an increment must be thought as part of the final system: it should be high quality, suitable for release, production-grade code and documentation; • Of course the above remarks are very important but: • You will not always get it right: changing a previous iteration’s code or design is inevitable and will happen all the time (the nature of the changes are important : are you re-designing all the time or just tinkering the code?); it is unrealistic to believe that the code of an iteration will never change during an entire project; • You must handle change during development;

10. Change is Inevitable because: • What was done previously is not perfect (this is very common); • The requirements were misunderstood (this is very common); • The requirements change (this is very common); • Change is common. The waterfall-way of dealing with this problem is to spend enough time during analysis to ensure that the requirements are perfect: (generally impossible). • The UP and all other iterative approaches (Agile or not) try to embrace change: • Each iteration looks at only one aspect of the project, does so quickly, and generates an executable at the end; early feedback can be provided; • Entire detailed requirements do not need to be frozen: each iteration and subsequent evaluation-feedback clarifies the requirements that were looked at in previous iterations and also sheds light on the requirements of the future iterations. • The requirements are clarified as we go along: a problem solving process.

11. Iterative, Evolutionary and Agile process • Of course, for this to work the resulting changes must not be major all the time: • We mustn’t make major blunders in our design; • Or completely misunderstand a requirement; • Or accept requirement changes (new feature please!) from external sources (client, marketing department etc.) that require a re-design. • In general, changes get harder to accommodate as the project progresses. • We must not fall into a chaotic process but properly managed iterations during which release-grade artefacts are produced can successfully accommodate change. Up to a point …

12. Fig 2.2 Iterative feedback leads to an evolutionary system closer to the ‘real’ requirements

13. Benefits of iterative development • Early progress that can be evaluated; • Early feedback via user engagement gives clarifications of requirements that will bring us closer to the ‘real’ requirements; • Complexity containment: we proceed by chunk; • Early tackling of risks (technical, requirements, objectives, usability etc.) • Late total project failure less likely; • Better productivity (especially when agile); • Less defects; • Possible process improvement within a single project

14. Potential problems of iterative development • Going into a prototyping-hacking mode; • Poor documentation; • Bad overall architectural design; • Unsuitable for some software: safety-critical (because safety properties must be proven true on the initial requirements); • Must strive for ‘good style’ all the time;

15. Each iteration should be time-boxed between 2 to 6 weeks: • Longer: rapid feedback and adaptation is lost, complexity of task is probably too great, falls into waterfall; • Shorter: quality and documentation will suffer; • If the manager feels that a deadline will not be met a recommended response is to de-scope : remove tasks or requirements from the iteration and schedule them to a future iteration, rather than slip the completion date (what do you think?)

16. The need for feedback and adaptation: • Feedback from the early demonstrations as client and developers try to understand each other to define and clarify the requirements as a team; • Feedback from testing implies that previous code needs to change; • Feedback from previous iterations to refine schedule and all other estimates (such as?); • Feedback from client and marketplace to change software emphasis (the marketplace does not stay still during software development …);

17. Overview of a simple example on 20 iterations (over a year) project: • Before the first iteration, hold an initial requirements workshop with client and senior developers. The UP Inception Phase: • Define the objectives of the project; • Do high-level requirements analysis by identifying use cases (e.g. 30) and features; • Identify non-functional requirements • Pick the 10% of the use cases that blend: • Architectural Significance; • High marketing value; • High risk this leads to say 3 use cases: UC2, UC11 and UC14; • Do a detailed analysis of the functional and non-functional requirements for these three use cases; • Review everything with everybody;

18. Ex. (cont.) • Then the developers alone hold an iteration planning meeting to define a subset of UC2, UC11 and UC14 for the first iteration to fit within a short (e.g. 4 weeks) timebox. Document the subset. Maybe tentatively schedule 2nd and 3rd iterations; • Iteration 1 (The UP Elaboration Phase): • Perform OOA documenting the Domain Model using UML; review by all developers (eg 3 days); • Perform OOD documenting the Design Model using UML; in pairs, review by all (eg. 2 days); • Start programming, integrating and testing using previous UML models as starting points; Try to update models; hold daily stand-up meetings (eg. 2 weeks); • One week before the end review and de-scope if necessary; • On Tuesday of the last week: code freeze to provide increment; • On Wednesday demo partial system to external stakeholders: get and document feedback; • 2nd requirements workshop: refine previous workshop document and detail another important 10-15% of the use cases

19. Ex. (cont.) • Iteration 2 : similar steps; • Repeat for four iterations and five requirements workshops so that at the end of iteration 4, perhaps 80% or 90% of the requirements have been written in detail but only 10% of the system has been implemented; • At the end of the Elaboration Phase, most of the requirements have been detailed by iterations and some of the system is available. All estimates should be reviewed • The UP Construction Phase can start: many more iterations, major requirements change is less likely during this period. Can be more ambitious on the amount of work to be performed. Keep learning and getting feedback; • In the UP the last phase is the Transition Phase: beta tests and deployment;

20. Fig 2.4: The Iterative Process

21. Risk-Driven and Client-Driven Iterative Planning • Decide which use cases to schedule in the elaboration phase, at the beginning of a project; select use cases according to: • Risks : to drive down the highest risks early on; • Client needs : to build visible features that the client cares most about; How does this apply to games development? • Risk-driven iterative development includes the practice of architecture-centric iterative development; early iterations focus on building, testing and stabilizing the core architecture: why?

22. What are Agile Methods? • Usually apply time-boxed iterative and evolutionary development, employ adaptive planning, promote incremental delivery, and include other values and practices that encourage agility ( rapid and flexible response to change). • Examples of practices include: • Common project workroom (from Scrum) • Self-organizing teams (from Scrum) • Programming in pairs (from XP) • Test driven development (from XP) While we will not have the time to review every technique we will keep an ‘agile’ point-of-view in our process. • The Agile Alliance has issued a manifesto and a set of principles.

23. The Agile Manifesto: We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.

24. The Agile Principles: • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. • Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. • Business people and developers must work together daily throughout the project. • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

25. The Agile Principles: (cont.) • Working software is the primary measure of progress. • Agile processes promote sustainable development. • The sponsors, developers, and users should be able to maintain a constant pace indefinitely. • Continuous attention to technical excellence and good design enhances agility. • Simplicity−the art of maximizing the amount of work not done−is essential. • The best architectures, requirements, and designs emerge from self-organizing teams. • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

26. What is Agile Modelling? • ‘The purpose of modelling is primarily to understand, not to document.’ • The waterfall calls for very detailed OOD documents that are handed to a programmer; • But modelling is still necessary; • The models (UML diagrams) created are also very good communication tools (face to face preferably); • Don’t model everything: simple issues can be handled at programming; • Use the simplest tools as possible: prefer sketching UML diagrams on whiteboards and capture the result with a digital camera; UML case tools can be used later to tidy things; • Do not model alone, or at least review by all; • Create models in parallel (e.g. interaction diagrams and class diagrams); • All models are likely to be inaccurate in some respect; • Best if whoever models is also responsible for coding;

27. Fig 2.5 Group-developed UML Sequence Diagram Sketch

28. What is an Agile UP? • The UP can be used in an Agile context: • Stick to the spirit of an iterative and evolutionary approach; • Continuously engage users for evaluation, feedback and requirements; • Continuously verify quality; test early, often and realistically; • Practice change request management and configuration management;

29. What are the UP Phases? • 4 major phases: • Inception : approximate vision, business case, scope, vague estimates; • Elaboration : refined vision, use cases, iterative development of the core architecture, resolution of high risks, identification of most requirements and scope, more realistic estimates; • Construction : iterative implementation of the remaining lower risks aspects; • Transition : beta tests, deployment; • This is not waterfall.

30. Fig 2.6 UP phases overview

31. What are the UP Disciplines? • The UP describes a set of on-going activities such as requirements analysis, design, testing, etc. • These are spread over all phases (and iterations); • The relative effort across these disciplines changes over time: e.g. early iterations tend to focus on requirements analysis rather than testing;

32. Fig 2.7 UP disciplines overview

33. Sample incep - transi - elaboration construction tion tion UP Disciplines Business Modeling Requirements Design Implementation ... ...

34. Inception Elaboration Construction Transition … Iteration … Iteration … Iteration … Iteration … Iteration 1 Use Case A Full … Use Case … Use Case A Simplified … … Use Case B Full … Use Case … Use Case … Use Case … Use Case … Use Case … Integration OOD OOA Coding & Testing

35. Adapting the UP • Apart form the fundamental principles (such as iterative and evolutionary, risk and client-driven development, realistic continuous verification of quality), the UP is very flexible. • We can write-up a Development Case document during inception to detail the choices of practices and artifacts that will be adopted for a project. • See Table 2.1, where ‘s’ stands for start and ‘r’ refine, for a partial sample Development Case.

37. You Know you did not Understand Iterative Development When … • You try to define all the requirements details before you start (or the entire detailed design; • You spend weeks doing UML modelling before programming; • You think that inception = requirements analysis, elaboration = design and construction = coding; • Your iterations are months long rather than weeks; • Your plans are fuzzy, overly detailed, document led and rigorously enforced; • People work in isolation, handing one document after another to the next person (e.g. requirements specification from analyst to a designer, design document to programmer, code to tester);

38. Conclusion • We will apply the UP, an iterative and evolutionary process, within an agile context. • Resources: • www.agilealliance.com (the Agile Alliance) • www.agilemodeling.com (Scott Ambler) • Craig Larman “Agile & Iterative Development: A Manager’s Guide”, Addison Wesley, 2004: • Questions?