Scaling Agile How to leverage Agile Techniques in the Enterprise

1. ScalingAgileHow to leverage Agile Techniques in the Enterprise Presented by: Bryan Campbell www.bryancampbell.com bryan@bryancampbell.com http://blog.bryancampbell.com

2. About the Presenter – Bryan Campbell More than 20 years of IT experience across a wide range of industries and organizations (government, public, private) The last 8 years focused on Agile and UP software development initiatives. Project and Program Manager of a wide range of small and very large projects. Extensive experience working in large scale enterprise environments (Telcos, Utilities, Software, Government, Finance) Recently the VP of Delivery Services for a consulting company specializing in agile software development. Currently, Sr. Program Manager at BMC Software Inc. Articles and blog can be found at www.bryancampbell.com

3. Agenda • Agile Explained • The Benefits of Agile • Challenges with Enterprise Adoption of Agile Techniques • Scaling Agile • The Future • Links

4. What Exactly is Agile? • Evolved in the last 1990s in response to the burdens of heavy documentation and frequent requirements change. • Essentially a ‘family’ of development processes such as Scrum, XP, Adaptive Software Development, Feature Driven Development, Crystal Clear and others (links on the last page) • Emphasizes (amongst other things) • face-to-face communication, • working software as the primary demonstration of progress, • emphasis on effective engineering techniques (Test Driven Development, Continuous Integration, Refactoring to Design Patterns), • frequent demonstrations of progress • Business and developer collaboration • Retrospectives and continuous improvement • Adopts a ‘it’s done when it’s done’ approach over excessive estimating and project completion predictions and has a ‘Just Do It’ feel.

5. How Has Agile Evolved? • Test Driven Development: “Write a Test, Watch it Fail, Write some Code, Watch it Pass”, rich collection of unit tests at the method level, code not complete until all tests pass. • Continuous Integration: Integration is hard and often delayed to just before the final build… if code is continuously compiled when code is checked in you can identify conflicts when they’re still easy to manage. • Refactoring: Application of design patterns to code to improve code structure, reuse and maintainability. • User Stories: Lightweight requirements capturing mechanism to speed the requirements gathering process and improve developer/BA communication. Agile has its roots in a collection of software engineering techniques which emphasized a more lightweight, quality driven approach to software development:

6. Overtime Agile Techniques Broadened Agile Manifesto (www.agilemanifesto.org) Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan Agile techniques began to enable other effective practices such as frequent demonstrations of working software, which encouraged greater customer collaboration and allowed more responsiveness to change. Eventually the Agile Manifesto was born:

7. Scrums, Pigs and Chickens One familiar adage used in Scrum projects is the differentiation between Pigs, committed project team members and Chickens, stakeholders not formerly part of the project team. During scrums only Pigs are allowed to talk.

8. An Agile Project Example Refactoring Continuous Integration Daily Stand-up Test Driven Development Product Owner User Stories Demonstration of Working Software Burndown charts

9. One of the first Definition of Agile Agile is an iterative and incremental (evolutionary) approach to Software Development which is performed in a highly collaborative manner by self-organizing teams with "just enough" ceremony that produces high quality software in a cost effective and timely manner  which meets the changing needs of its stakeholders. - Scott Ambler, www.agilemodeling.com , Managing Agile Projects

10. Agile = Fragile • Many of the initial Agile engineering principles and the Agile Manifesto were greeted warmly at the grassroots developer level. It enabled developers to work in ways they hadn’t experienced before. • However, Agile was ‘silent’ on many of the realities facing widespread adoption, specifically: • Architecture • Globally distributed teams • Large scale initiatives • PMP related disciplines (budgets, resources, communication, schedules, scope) • Transition (ITIL) activities • Agile was also struggling under the weight of its own success, more people with less experience were applying these techniques and the result was a number of failing ‘Agile’ projects.

11. Agile – Less Engineering More Enabling • Overtime, Agile has begun to emphasize many of the ‘people-centric’ aspects of Scrum and less of the engineering practices (although these are still good and useful). • Lean and Six Sigma techniques began to infuse themselves into Agile practices include: • Continuous Improvement: Japanese ideas like Kaizen began to appear in the form of retrospectives, the entire team regularly discussing what worked and what didn’t work in an iteration. • Focus on the Value Chain: Extending agile techniques, particularly Pull, across each segment of the value chain. • Business Value Emphasis: Understanding the business value of work and prioritizing effort around this work.

12. The Agile Core • Hallmarks of Agile Project Techniques : • Prioritized Backlogs Drive Release Dates: Features are managed in a backlog, prioritized by business value and releases are determined by the development velocity and what is deemed acceptable as a production release. • Tradeoffs are an important part of the process: Prioritizing business value of work with Business and IT working collaboratively to balance risk. • Sustainable Pace: Team members are involved in estimates and commitment dates. Overtime, weekend work and heroic efforts are ‘bad smells’. • Frequent demonstrations of working functionality: The project should be broken into a series of time-boxed, iterations that have a demo to show progress to all stakeholders. • Continuous Improvement: Weekly team retrospectives should be held to learn how to improve and enhance the project delivery efforts. • Communication: Daily ‘scrums’ should be held amongst team members to understand what has been accomplished, what will be accomplished and what roadblocks exist. • Visibility: Information on project status, progress and issues/risks should be maintained in real-time, web accessible tools. • Quality: Everyone owns ‘quality’. Standards, Test automation and key principles such as Test Early / Test Often are emphasized. 12

13. Agile Emphasizes • Agile projects have several core attributes: • Requirements are managed in a lightweight fashion, usually with ‘user stories’, short business narratives of functionality (sometimes described as a ‘place holder for a conversation’ between business users and developers). • Requirements form a backlog which is prioritized by the business users. A Release backlog represents enough functionality that the system can be deployed into production. • Empowered and co-located teams. • Iterations are used to demonstrate functionality to business stakeholders and elicit feedback, iterations are timeboxed. • Based on progress across iterations, velocity is determined which illustrates how rapidly the team is progressing. • Velocity is mapped against the backlog and shows a burn-down for the release. • Team retrospectives at the completion of each iteration to review what went well and what didn’t go well. • Daily stand-up meetings where each team member shares What they Did, What They will Do and what obstacles/roadblocks they’re facing. • A ScrumMaster enables and guides the teams without being prescriptive or commanding.

14. Agile De-emphasizes Big design and requirement gathering ‘up-front’. Predictions on project completion. Death march projects where project teams make up the difference for poor estimates with unpaid overtime. Use of tools that force behaviors (task management tools, requirements management tools etc.). Top Down Management/Control. “Heavy” documentation, particularly status reports, Software Architecture Diagrams, Software Requirements Specifications, Test Plans etc.

15. The Benefits of Agile This all sounds great… what’s the problem? Emphasizes many of the ‘right’ things: collaboration, team empowerment, working software, frequent demonstrations of progress etc. Lightweight, relies on whiteboards, index cards and facilitation techniques (planning poker, Fist of Five etc.) Very appealing to developers who make up a large portion of most development teams with its development focus. Business sponsors like the idea of time-to-market opportunities and driving the features of the development lifecycle. Implicitly focuses on ‘push vs. pull’ Simple and easy to understand. Contemporary and ‘modern’

16. Agile Works Well in Specific Situations Unfortunately this is a small subset of most Enterprise IT organization portfolios Smaller projects with smaller teams (5-8) Highly skilled teams Co-located teams Lower risk initiatives (internal systems, enhancements etc.) Advanced technologies (object oriented languages primarily) High Novelty / High Change requirements settings

17. Challenges with Enterprise Adoption of Agile Techniques • Little guidance for managing beyond the ‘Design and Implementation’ disciplines e.g. • Budget/Resource/Communication/Scope/Risk Management • Enterprise Architecture • System/Integration Testing • Deployment (coordination with other releases, release windows) • Most organizations need more confidence on dates and costs before they can commit to a project. • Larger IT Enterprises often have widely distributed teams and business units • Publically traded companies in the US and heavily regulated industries have mandatory documentation/review and testing processes

18. Scaling Agile • Difficult to be completely prescriptive but there are areas of guidance that are valuable. • Start with an appropriate ‘meta-model’ that approach the project in distinct phases with clear exit criteria • The Rational Unified Process is very good • Variants like OpenUP, Enterprise Unified Process are also very good • Ensure well defined roles and recognize roles outside of the development team that are needed • Assess project risk using a framework like that supplied by Boehm and Turner in Balancing Agility and Risk: A Guide for the Perplexed • Ensure solid commitment and buy-in at senior leadership levels • Can’t overemphasize enough • Ensure clear understanding of the trade-offs

19. Determining an Agile vs. Plan Driven Approach • Plan-driven • home ground: • Highcriticality • Juniordevelopers • Requirementsdon'tchangetoo often • Large number of developers • Culture that demands order Agile home ground: • Low criticality • Senior developers • Requirements change very often • Small number of developers • Culture that thrives on chaos

20. Plan and Estimate Planning is necessary for large scale projects. This includes Release Plans, Iteration Plans, Schedules with Milestones/Dependencies, Resource Plans. Differentiate between Estimates (wide confidence level) and Commitments (narrow confidence level). Ensure appropriate communication and reporting (Status Reports) against plans. Familiarize stakeholders and teams withthe Cone of Uncertainty.

21. Manage Risk • Understanding and managing risk can be the difference between success and failure, some guidelines that RUP emphasize that strengthen agile techniques include: • Do the hardest things first • Start developing as early as possible • Mitigate risks explicitly by identifying stories and iterations they will be addressed • Don’t leave Elaboration without risks being mitigated • Prioritize using a Project Troika • Ensure a balance between architectureand functionality • Project Manager, Architect and BusinessAnalyst

22. Distributed Teams • Where geographically distributed teams are necessary remember that co-location is ‘good’ so optimize around this as much as possible • Have development teams focus on ‘Feature’ sets • Keep Feature Teams as ‘agile’ as possible • Ensure teams receive sufficient detail to build what is required: • Architecture • Requirements • User Interfaces • Ensure that teams are enabled by an effective ‘ScrumMaster’ to escalate issues and resolve roadblocks • Ensure retrospectives occur at the team level at the end of iterations • Support teams with a Program Team to provide overall guidance/direction • Establish an Architecture Team which is responsible for ‘feeding’ Feature teams with frameworks, components and guidelines.

23. Emphasize Architecture The Architecture team under guidance of the Software Architect builds difficult or critical components of software that can be referenced by the development teams in their iterations. The Feature team architects work closely with the Architecture team and ensure/review the quality of the code developed by the programmers. Architecture Project Support Developers Architecture is too important/complex/expensive to most large organizations to rely on an ‘emergent’ form. Ensure architecture and design considerations are understood early in the project. Associate stories to architecture requirements. Document architecture stories and ensure they are prioritized appropriately in the backlog. Establish an Architecture Team that can build frameworks and components for the Features Teams to leverage.

24. Effective Requirements Management Large projects need more than index cards with one or two sentence user stories. Ensure a web-accessible, requirements tool is available. Put someone in charge who is responsible for all requirements Leverage Models and Executable Requirements as much as possible. Agile terms like ‘Placeholder for a conversation’ need to be backed up with documentation.. This can be lightweight and it can be done in real-time! Balance breadth and depth: Ensure basic flows of requirements are fulfilled and worry about exceptions later.

25. Post-Agilism or Pragmatic Agilism • A growing movement is beginning to recognize that there is more to agile development than many of its more dogmatic members proclaim. • Jonathan Kohl is one of its proponents and describes Post-Agilism as: A growing movement of former Agilists who have moved beyond Agile methods, using a wide variety of software development tools and methodologies in their work. • Lean software development is also seeing considerable interest as it emphasizes ideas such as minimizing waste, deliver as fast as possible, build integrity in and empower the team.

26. The Future • This was a very high level overview on Agile and techniques for effectively Scaling Agile to leverage its benefits in the future. • Agile does have many benefits but it needs to applied differently in larger enterprise environments. • More discussion and exploration is required. • Let’s discuss our own experiences

27. Agile Development Links Methodologies • Scrum • Crystal Clear • Extreme Programming • Adaptive Software Development • Feature Driven Development • Dynamic Systems Development Method • OpenUP • Enterprise Unified Process • Lean Software Development Techniques • Planning Poker • Fist of Five Reference Information • Agile Wikipedia Reference • Post-Agilism • The Project Troika

28. Bonus: Agile Collaboration Tools An area of interest I’ve been exploring lately has been how to increase the sense of ‘co-location’ without being co-located. New Web 2.0 and Project Collaboration containers offer some interesting opportunities. Focus on communicating ‘activities’people are involved in which differsfrom typical PM focus on ‘outcomes’ Activities convey more about what people‘do’ … most people ask questions about “what did you do today” or “what are you doing” ‘Swarm’ theory starts to take hold as the collective team becomes aware of what everyone is working on in real-time Some additional detail can be found in an article called Lean Scrumming