Agile Development

1. Agile Development ARKANSAS BLUE CROSS BLUE SHIELD RSFOX@ARKBLUECROSS.COM (501) 396-4160 Tuesday 9/25, 8:40 - 9:20

2. Robert Fox - Data Architect, Arkansas Blue Cross Blue Shield 17 years of data architecture and warehousing experience in the finance, telecom, and health insurance industries. Installed over 50 data warehouses worldwide, some loading more than 4.5 terabytes of new transactional data per day. Guest lecturer in information management masters degree programs at 3 universities. Speaker at national conferences for Oracle, Fidelity, and MDM-CDI.

3. Abstract - Agile IT is about finding balance • The term “Agile Development” is used in several ways. Most commonly, the phrase is used in a narrow sense to refer strictly to the development phase of a software development project. In this presentation, I will be using the phrase in a much larger sense: creating an IT organization that is agile in ALL of its parts. • I think the term “Agile” itself is misleading. According to the dictionary, “agile” means “nimble, quick, and flexible”. This definition makes it appear that this “agile” thing we are seeking is an extreme, the opposite of “rigid.” Many of the proponents of agile development take this view. I believe, however, that agility is not an extreme, but a balancing act between extremes: • balancing the desire for continual user feedback with the need to let developers remain focused • balancing the need for predictability with the need for flexibility • balance the development of strategic, technical infrastructure with the need to respond to immediate business requests • balancing the need to provide self-service interfaces with the need for controlling the quality of data and processes • All too often, organizations adopting “agile development” completely abandon their existing processes (which were unbalanced in one direction) in order to adopt an unbalanced polar opposite process. • Ideally, mature agile development is about finding the correct balance across the entire organization.

4. AgendaBalancing the Building Blocks of an Agile IT Organization Organization Structure Project Management Methodology Architecture Development Methodology Self Service

5. Agile Development MethodologyBalancing the need for continuous feedback with the need to keep developers focused Organization Structure Project Management Methodology Architecture Development Methodology Self Service

6. Traditional Developmenta.k.a. “waterfall,” or “release” • PRO’s • Supported by most project management software • Very clear, up-front understanding of entire timeline • Clear lines of responsibility Analysis Design Develop Test • CON’s • Sequential • Not iterative • Requestor not involved in most of the process • Long time between request and delivery Implement • Works well for: • Large, complex projects involving many inter-related components or departments • Projects which are foundational to future projects

7. “Agile” Development • PRO’s • Very rapid response to changing needs • Requestor involved throughout process Time-boxed scope Cross-functional development • CON’s • No clear insight into long-term timeline, resource needs, cost, etc. • Not well suited to most project management tools • Difficult to coordinate strategy and architecture across sprints and projects • Responsibilities less clear • Works well for: • Small projects which can be completed in isolation from other resources and projects • Projects which, while perhaps critical on their own, are not critical to future projects • Small development shops where everyone is working on the same project

8. This Is Really Two Issues • Finding the right balance of interaction between requestor and developer • Finding the right balance of administrative effort necessary for management insight Lets think about them separately…

9. Balancing Levels of Interaction During Software Development • Working in teams • Can keep work from being done wrong, but can also keep any work from being done at all. • Working in isolation • Can keep people focused, but they can be focused on the wrong things

10. Finding the Balance • We need isolation to focus, but we need to come together for vision. • We need to get together as soon as there is a question, not at the next weekly meeting. • We need to develop an environment where people are not reluctant to get out of their cubes and go sit down with someone else to work things out. • Identify your best communicators • Give them a title that recognizes and encourages this skill • Make sure they are assigned to critical projects • Use them to mentor other members of the team

11. Agile Project ManagementBalancing the need for management insight with the need to minimize project overhead Organization Structure Project Management Methodology Architecture Development Methodology Self Service

12. Why do YOU need PM? The questions isn’t as simple as you might think. PM is absolutely necessary for management insight, but in many cases it has become a tool for looking over peoples shoulders because you don’t trust them to do their jobs. Ask yourself, “If we backed off on project management, what’s the worst that could happen?”

13. Is Oversight Needed Because of Cowboy Developers? Will the developers play with technical toys if you don’t have PM oversight? Ask yourself it they are they willfully violating standards, or is the problem that there are no standards? Maybe you told them “what” the standards are, but never told them “why” these standards are needed Most people will do the right thing if they know what it is, and agree that it’s right in the long run, and it’s not too much harder in the short term.

14. “Doctor Fun” by David Farley, October 2002

15. Business Goals versus Natural Desires “Do what I tell you to do, or else…” “If you do things this way, here’s what will happen…” Job Security Recognition Strong Commitment Business Goal Business Goal Social/Fun Weak Commitment Family Creativity It could be that the reason your organization needs so much PM Oversight is that your people haven’t bought into the vision. Most people will do the right thing, if they understand what and why in terms that are personally meaningful

16. Finding the Balance • Clearly document standards and processes • Communicate the vision of the future made possible by these standards, and put in terms relevant to the target audience • Invest in the tools and infrastructure necessary to make following the standards easier than violating them

17. Oversight Needed Because of Weak Supervisors? Are the supervisors: • Incapable of adapting to changing conditions, and you need PM oversight to steer their projects for them? • Incapable of locking down scope creep? • Incapable of enforcing standards? In many cases, this is due to the same problem of communication of standards and vision. In some cases, however, it is not.

18. Agile Organization StructureBalancing the need for creative thinking with the need for well-defined, repeatable process Organization Structure Project Management Methodology Architecture Development Methodology Self Service

19. Predictabilty vs Flexability • Two Approaches to Project Management • The Factory Floor • Research and Development • Balancing Lessons From Jack • Understand which processes are “cookie-cutter” and which are not, and manage them differently • Put the right people in the right roles Jack Welch – CEO of GEFortune Magazine “Manager of the Century”

20. Maybe it’s not weak supervisors, but a weak organization chart • Creative Roles • Development • Strategic Planning • User Interface Design • Training • Documentation • Tool Selection • Solution Oriented • Quality Roles • QA/Testing • Change Management • Security • Operations • Audit • Compliance • Process Oriented Are the supervisors incapable of adapting to changing conditions, and you need PM oversight to steer their projects for them? • Perhaps they are in the wrong positions • Operations – conservative, process oriented • Development – creative, solution oriented • Perhaps they need to be let go • Jack Welch fired the bottom 10% of his managers for several years

21. Agile Development Can Be an Organizational Issue Project Flow Often, development is not agile because the same resources are asked to focus on too many things. Organization should reflect functional responsibility. We know better than to let developers test their own code. But do you know to separate creative development from rigid operations? To separate tactical development from strategic architecture?

22. Know when to hold ‘em, and know when to fold ‘em… • Was it an issue with communication of standards or vision? • Was it an example of misplaced skillset? • Or was it a case where the person needs to be let go? Adding permanent project management overhead should be a last resort, not a first.

23. Agile ArchitectureBalancing the need to invest in tomorrow’s infrastructure with the need to meet the demands of the business today Organization Structure Project Management Methodology Architecture Development Methodology Self Service

24. Today versus Tomorrow Strategic Determining long range destination and planning a path to it Tactical Keeping the lights on. Meeting day to day business demands These should be different roles Navigating Piloting

25. Finding the balance How do you balance the need to get things done quickly and cheaply with the need to prepare for the future? • Dedicate resources to developing and maintaining a 3-5 year strategy for ALL the functional areas of your organization. Include both what the future looks like, and a tentative roadmap to get there. • During budgeting process each year, make sure to ask for funds for any IT-sponsored, strategic projects. • When tactical business requests come in, analyze each to see how it can be implemented in a way that aligns with the long term strategy.

26. Agile Business IntelligenceBalancing the need to remove development bottleneck with the need to ensure data is used appropriately Organization Structure Project Management Methodology Architecture Development Methodology Self Service

27. Agile Means “IT is Not a Bottleneck” “Enterprises where IT addresses more than 20% of BI requirements will continue to see the snowball effect of an ever-growing BI requests backlog” Boris Evelson - Forrester Research “BI requirements change faster than an IT-centric model can keep up” Jim Ericson - Editorial director of Information Management Maybe what we should be looking at isn’t turning around IT projects faster, but instead architecting a managed platform where the business is able to self-serve their information needs using “safe” information.

28. Finding the balance • Implement business rules in the database, not in business reports • Build specialized data marts for specific data domains • Provide awesome meta data • Use tools that can explore prebuilt dimensional cubes, and drill through to prebuilt detail reports • Clean up your data • Make it easier for your users to user your vetted data than to try to pull it themselves • Stamp out private copies of data

29. Summary – Agile IT OrganizationsBalancing the need to remove development bottleneck with the need to ensure data is used appropriately Organization Structure Project Management Methodology Architecture Development Methodology Self Service