Agile Systems Development

1. Agile Systems Development Neil McBride

2. Agenda What’s the problem? What’s the metaphor? – Building Construction versus Jazz Improvisation Agile Methods Examples Agile Manifesto XP as an example of agile method Four Values Basic Principles Planning games, pair programming and metaphor. Critique Neil McBride Centre for IT Service Management Research

3. The Problem Failure of software project is endemic. Schedules slip, Projects are cancelled, Business needs never met, Changes made faster than system can be developed, Testing is inadequate, Quality not managed, Staff turnover, Systems become too large to manage. Neil McBride Centre for IT Service Management Research

4. Computer systems cannot cope with change. Difficult to change, Cannot adapt to organisational change, Require certainty. But organisations’ and businesses’ problems change New customers, Transactions changes, New products, Demands of Internet-based systems Neil McBride Centre for IT Service Management Research

5. How do we deal with change? Large scale project management approaches try to: Reduce change Increase certainty Eliminate risk by planning, documentation and contracts. We do what it says on the contract We freeze requirements and we formalise such freezing though stage meetings. Changes become rewrites. Our metaphor is the construction industry and our project management practices are drawn from the industry. What if the metaphor is wrong? Neil McBride Centre for IT Service Management Research

6. The Construction Analogy Assumption: sequential development process, predictable and stable environment, goals of achieving efficiency and reducing uncertainty Structure IS stable artifacts? Requirements ‘fixed in concrete’ IS as social artifacts Process Linear? Formal join points? SSADM-like? Roles Cultural ghettos, divergent, conflict and formalisation. Blinkered view. Neil McBride Centre for IT Service Management Research

7. Film Production Script based on literary sources Revision during production Interpretation by different creative artists Pre-production, Filming, Post-Production Neil McBride Centre for IT Service Management Research

8. Structure More pliable product Builds on body of work But little active role of users. Process Dynamic development Greater scene of teamwork, changing roles, continuous involvement of many creative staff. Roles Variety of roles. Wide-skill set. Some role changing. Neil McBride Centre for IT Service Management Research

9. Jazz Improvisation Use of minimalist musical structures including harmonies, melodies and rhythm. Small team elaborating on simple structures in complex ways Musicians operate with a set of social norms, with changing roles and intense interaction. Minimalist Structures Constrain the turbulence of the jazz process by specifying particular ways of inventing and co-ordinating musical ideas’ Neil McBride Centre for IT Service Management Research

10. Neil McBride Centre for IT Service Management Research

11. Structure Dynamic systems, subject to change in response to organisational change Minimal componentised structures Patterns Process Variation with socially determined process structured Iterative development and continuous delivery Theme development Role Role rotation. Importance of mentors Continuous communication and listening. Neil McBride Centre for IT Service Management Research

12. Neil McBride Centre for IT Service Management Research

13. Agile Software Development Approaches Ancestry in Rapid Application Development Held up as antithesis of traditional software development. Focus on: Early delivery priority business requirements. Dealing with partial knowledge Reduced documentation Small groups of programmers Iteration Continuous testing Integral customer involvement Neil McBride Centre for IT Service Management Research

14. The Agile Alliance Snowbird 2001 : Meeting of representatives of agile methods Purpose to get all the leaders of lightweight methods into one room ‘Define a developer community freed from the baggage of Dilbertesque corporations.’ [Respond to] ‘the failure of the standard "fixed" process mindset that so frequently plagues our industry.’ ‘The Agile movement is not anti-methodology, in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment.’ Neil McBride Centre for IT Service Management Research

15. Agile Methods Adaptive Software Development Feature Driven development Crystal Clear Method Dynamic Systems Development Method Rapid Application Development (James Martin) Scrum Pragmatic Programming Extreme Programming Neil McBride Centre for IT Service Management Research

16. Agile Alliance Values Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan Neil McBride Centre for IT Service Management Research

17. Principle 1 Our highest priority is to satisfy the customerthrough early and continuous deliveryof valuable software. Software as the business output. Continuous delivery provide regular feedback. Enables customer evaluation Neil McBride Centre for IT Service Management Research

18. Principle 2 Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Promoting software ‘evolvability’ But still avoiding radical changes at the end of the development project. Neil McBride Centre for IT Service Management Research

19. Principle 3 Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Rapid feedback. Daily build Requirements can be tested and altered quickly. Key importance of iterative and incremental development From fixed pricing to adaptive pricing? Defining releases to match business deadlines and user ability to absorb changes. Neil McBride Centre for IT Service Management Research

20. Principle 4 Business people and developers must work together daily throughout the project. Role of developers changing towards business orientation. Development managers with both business and technical understanding and still writing code Neil McBride Centre for IT Service Management Research

21. Principle 5 Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. Agile approaches put emphasis on people factors – sociability, talent, skill communication. ‘Personnel attributes and human relation activities provide by far the largest source of opportunity for improving software productivity’ Barry Boehm XP very demanding of people skills. We’re not all Kent Becks or Ward Cunninghams! Need for extraordinary skills, strong tacit knowledge and discipline Neil McBride Centre for IT Service Management Research

22. Principle 6 The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Stand-up meetings. Informal communication Tacit knowledge transfer Role of written communication. Problem of unrecognised shortfalls of tacit knowledge Neil McBride Centre for IT Service Management Research

23. Principle 7 Working software is the primary measure of progress. Early delivery Measurement. .. Not lines of code Measurements of what? Functions? Passed tests? Neil McBride Centre for IT Service Management Research

24. Principle 8 Agile processes promote sustainable development. The sponsors, developers, and users shouldbe able to maintain a constant pace indefinitely. Workaholism. Ineffectiveness of work-all-hours mentality. Creativity needs recreation Sustained overtime is a bad sign Neil McBride Centre for IT Service Management Research

25. Principle 9 Continuous attention to technical excellence and good design enhances agility. Refactoring and personal quality requirements Expertise and attitude of the programmer Neil McBride Centre for IT Service Management Research

26. Principle 10 Simplicity--the art of maximizing the amount of work not done--is essential. Avoiding bells and whistle. Simple solutions are easy to understand Who decides what is simple? Neil McBride Centre for IT Service Management Research

27. Principle 11 The best architectures, requirements, and designs emerge from self-organizing teams. Self-governing teams Emergent behaviour The ‘jelled’ team Complexity Theory and Post-modernism Neil McBride Centre for IT Service Management Research

28. Principle 12 At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Value of organisational learning. Time cost of review Neil McBride Centre for IT Service Management Research

29. eXtreme Programming Now let’s see the Agile Manifesto Principles in Action. Comes from a group of American programmers : – Kent Beck Ward Cunningham Ron Jefferies. Assumes an object-oriented approach. Closely connected with the patterns movement. Neil McBride Centre for IT Service Management Research

30. Extreme Programming Little documentation. Source code is only documentation No software specification No separate design and testing phase Design for change prohibited No formal reviews or inspections Neil McBride Centre for IT Service Management Research

31. Extreme Programming A view of what development is. Four values. Some basic principles. Twelve practices. An approach to development .. The development episode. Neil McBride Centre for IT Service Management Research

32. What’s important in software development management? Cost Time Quality Scope What’s important in the process? Coding! Testing Listening Designing. Neil McBride Centre for IT Service Management Research

33. Extreme Programming: Four Values Communication Making communication essential. Face-to-face. Environment. Pair programming. Simplicity What is the simplest thing that can possibly work? Removing complexity (by refactoring) Challenging complexity (by continuous integration, pair programming) Feedback Coaching. Testing. Starting with test case. Daily integration. Quick releases. Courage. Make a decision. Team support. Ready to start again. Neil McBride Centre for IT Service Management Research

34. Extreme Programming. Fundamental principles. Rapid Feedback. Learn immediately. Stimulus/response. Days not weeks/ months. Assume simplicity. Sort today’s problems today. No belts and braces. Using tests. No design for reuse. Incremental Change. Series of small changes. Evolve solutions. Plan, design, team, change a little at a time. XP adopted a little at a time. Embrace Change. No frozen specs. Quality work. How. Constant review (pair programming) refactoring. Removing redundancy. Pride in craftsmanship. Art.. Others (see for yourself) Teaching learning, Small initial investment, play to win (not (not to loose)), Concrete experiments (leave no abstract decisions untested), open, honest communication, Work with people instincts, accepted responsibility, local adaptation, travel light, honest measurement. Neil McBride Centre for IT Service Management Research

35. Extreme Programming : Basic Practices The Planning Game - Developers and customers scope and plan using story cards Small releases - Release as small as sensibly possible. Metaphor – Identify overarching metaphor e.g. contracts and contract management, tracking robot. Simple design Design as simple as possible at any given moment. Only leave whatever is really needed. Eliminate complexity Testing – Write the test first. These must run flawlessly for development to continue. Refactoring – restructuring system without changing its behaviour. Pair Programming – All production code is written with two programmers at one machine. Neil McBride Centre for IT Service Management Research

36. Extreme Programming : Basic Practices Collective Ownership – Anyone can change code anywhere in the system at anytime. Opposite of Software Configuration Management. Works if there’s continuous integration, simple writing, complete visibility. Continuous integration. Integrate and build system many times a day, every time a task is complete. 40-hour week. Never two weeks in a row overtime. On-site customer – real live customer (not line manager) sitting with team all the time. More value out of system with more business contribution. Coding standards – programmers write all the code in accordance with rule emphasizing communication though the code. Neil McBride Centre for IT Service Management Research

37. Extreme Programming : Roles Programmer – adds actual value, at heart of development Customer – importance of customer engagement Tester – helps customer write functional tests Coach – responsible for process as a whole – XP expert. Provides toys and food. Tracker – conscience of XP team. ‘Is the feedback loop being closed? Consultant – ‘wizard’ supplies deep technical knowledge. Big Boss – Champion of development team. Neil McBride Centre for IT Service Management Research

38. The Planning Game Business writes stories. Estimate time for each story. Break down stories if necessarily. Clarify with customer. Sort stories and decide what’s in the first iteration. Sort on value and risk. Turn stories into tasks. Neil McBride Centre for IT Service Management Research

39. Programmer Accepts tasks and estimates. Finds a partner. Writes test cases (which will all fail at the start because there’s nothing there). Gets all tests working. Get something, probably with many small classes, up and running as fast as possible. ‘Go like the clappers’ Minimalist design. Neil McBride Centre for IT Service Management Research

40. Critique Origins – OO, Patterns movement, key group of ‘gurus’ Refactoring – programmer psychology. Programmer versus manager. Business / IT gap approach to crossing it? Scalability Debate. Size does matter. Reaction to outsourcing and ‘big thump’ delivery of outsourcing. Philosophy: Chaos and Complexity, Emergence, Post-modernist, Eastern Philosophy. Culture . ‘Biggest barrier to XP is culture’ Beck. Precedence of code as the lingua franca. A programming high priesthood? Neil McBride Centre for IT Service Management Research

41. Bibliography and Sources Beck, K. (1999) Embracing change with extreme programming. IEEE Computer, October 1999, 70-77. Beck, K. (2000) Extreme Programming Explained. Addison Wesley Agile Methods List (CERN) Agile Software Development Ecosystems Build better software Agile Alliance Questioning Extreme Programming Agile Software Development in Theory and Practice (MSc Thesis, Finland) Martin Fowler articles on XP and Agile Methods Boehm, B and Basili, V. Gaining Intellectual Control Over Software Development Neil McBride Centre for IT Service Management Research

42. Bibliography and Sources Jim Highsmith Links and resources Xtreme programming.com (Ron Jefferies) Extreme Programming Roadmap (Ward Cunningham) Scrum Crystal Clear Extreme Programming Case Study - Teaching Extreme Programming in a university DaimlerChrysler C3 Case Study On the Productivity of Agile Software Practices: An Industrial Case study Hi-Tech Workplace no better than factories (BBC News) Avison, D and Wilson, D. (2001) A viewpoint on software engineering and information systems: what we can learn from the construction industry. Information and Software Technology 43, 795 - 799. Kamoche,K and Pina e Cunha, M. (2001) Minimal Structures: From Jazz Improvisation to Product Innovation. Organisational Studies 22(5) 733 - 764. Neil McBride Centre for IT Service Management Research