USC-CSE Annual Research Review

1. USC-CSE Annual Research Review Los Angeles, CA March 17, 2004

2. Outline • Background • Impact of 2003 Workshop on book • Definitions and characterizations • Home grounds and the five critical dimensions • A risk-based approach to balancing agility and discipline • Observations

3. Background (Rationale for book) • Two approaches to software development • Plan-driven (SW-CMM, document-based, heavy process) • Agile (XP, tacit knowledge, light process) • Both have strengths and weaknesses • Agile and plan-driven proponents are believers • Many of us are perplexed • We still perceive perplexity • More voices being heard: • Larman • Anderson • Rosenberg/Stevens • Poppendieck • Others???

4. Impact of 2003 Workshop - 1 • “Disciplined” changed to “plan-driven” • Driven by process and product plans • Both agile and plan-driven methods need to balance agility and discipline • XP: 12 highly-disciplined practices • RUP: Flexible mix of requirements, design, code

5. Impact of 2003 Workshop - 2 • More examples of hybrid methods • AgileTek: Agile with top-level plans, architecture • Lightweight versions of RUP and TSP • XP variants: • Part-time customers • Longer workweeks • Less collective ownership and metaphor • More examples of empirical data • Balancing illustration and need for transition guidance • Collins’ – Good to Great

6. Hierarchical Organization Great Organization High Culture of Discipline Bureaucratic Organization Start-up Organization Low Ethic of Entrepreneurship Low High Collins’ Good to Great

7. Trends since last year • Increasing demands for agility, velocity, quality • Alternative manifestos • Push toward evolutionary acquisition in traditional communities • More agile approaches to CMMI improvement, assessment • More use of agile methods in large companies • More hybrid agile/plan-driven approaches

8. Outline • Background • Impact of 2003 Workshop on book • Definitions and characterizations • Home grounds and the five critical dimensions • A risk-based approach to balancing agility and discipline • Observations

9. Sources of Perplexity • Distinguishing method use from method misuse • Claiming XP use when simply “not documenting” • “CMM Level 4 Memorial Library” of 99 2-inch binders • Overgeneralization based on the most visible instances • XP is Agile; CMM is Plan-driven • Multiple definitions • Quality: customer satisfaction or compliance? • Claims of universality • The pace of IT change is accelerating and Agile methods adapt to change better than disciplined methods therefore Agile methods will take over the IT world. • Software development is uncertain and the SW-CMM improves predictability therefore all software developers should use the SW-CMM

10. Sources of Perplexity (2) • Early success stories • Chrysler project that successfully birthed XP was later cancelled • Cleanroom has never made it into the mainstream • Purist interpretations (and internal disagreements) • “Don't start by incrementally adopting parts of XP. Its pieces fit together like a fine Swiss watch“ • "An advantage of agile methods is that you can apply them selectively and generatively." • "If you aren't 100% compliant with SW CMM Level 3, don't bother to bid" • "With the CMMI continuous interpretation, you can improve your processes in any order you feel is best."

11. Key Definitions • Agile method – • one which fully adopts the four value propositions and twelve principles in the Agile Manifesto. • Discipline – (per Webster) • control gained by enforcing obedience or order; • orderly or prescribed conduct or pattern of behavior; • self-control. • Plan-driven – • Processes driven by documented plans (order is often defined in plans) • Plan – (per Webster) • a method for achieving an end (a process plan); • an orderly arrangements of parts of an overall design (a product plan). • We assume that such plans are documented.

12. Finding Middle Ground • A pragmatic view • Both approaches have “home grounds” • A broad range of environments and needs • Trends require better handling of rapid change • Processes should be the right weight for the job • We believe risk-based analysis is the key to finding middle ground

13. Contrasts and Home Grounds • Comparison is difficult, but we found 4 areas where there are clear differences • Application • Project goals, size, environment; velocity • Management • Customer relations, planning and control, communications • Technical • How the software is developed • Personnel • The type and competency of developers and stakeholders

14. Application Characteristics • Primary goals • A: Rapid value, responsiveness to change • P: Predictability, stability, high assurance • Size • A: Small to medium • P: Large • Environment • A: Turbulent, high change • P: Stable, requirements defined • Project velocity • A: Code, learn, and fix • P: Architect for parallel, incremental development

15. Management Characteristics • Customer Relations • A: Dedicated, collocated; where feasible • P: Contractual; increasingly evolutionary • A: Trust through working software and participation • P: Trust through process maturity evaluations • Planning and Control • A: Means to an end • P: Anchor processes, communication • Project Communications • A: Tacit, interpersonal knowledge • P: Explicit, documented knowledge

16. Technical Characteristics • Requirements • A: Adjustable, informal stories • P: Formally baselined, complete, consistent specifications • Development • A: Simple Design • P: Architecture-based design • Testing • A: Automated, test-driven • P: Planned, requirements-driven

17. Technical Characteristics (2)Cost of Change: Beck, Li Beck Li Li

18. Changes in High-stakes Constraints Cost of Change Most Changes Time Technical Characteristics (3) Cost of change: Poppendieck • At least two cost-escalation curves • Lean development: Architect to defer high-stakes decisions – e.g. COTS • Easier to change an unmade decision

19. Technical Characteristics (4)Cost of Change: TRW Beck Li Li

20. Technical Characteristics (5)Cost of Change: CCPDS-R Beck

21. Technical Characteristics (6)How Much Architecting? Beck 10,000 KSLOC Sweet spot 100 KSLOC Liu Liu 10 KSLOC

22. Personnel Characteristics • Customers • A: CRACK customers throughout development • P: CRACK customers early • CRACK: Collaborative, Representative, Authorized, Committed, and Knowledgeable • Developers • A: Heavy mix of high caliber throughout • P: Heavy mix early with lower capability later • Culture • A: Many degrees of freedom (Thrives on chaos) • P: Clear policies and procedures (Thrives on order) • Collocated customers are difficult to find/keep/keep current on either side

23. Personnel Characteristics (2)Modified Cockburn Levels Level Characteristics 3 Able to revise a method (break its rules) to fit an unprecedented new situation 2 Able to tailor a method to fit a precedented new situation 1A With training, able to perform discretionary method steps (e.g., sizing stories to fit increments, composing patterns, compound refactoring, complex COTS integration). With experience can become Level 2. 1B With training, able to perform procedural method steps (e.g. coding a simple method, simple refactoring, following coding standards and CM procedures, running tests). With experience can master some Level 1A skills. -1 May have technical skills, but unable or unwilling to collaborate or follow shared methods.

24. Outline • Background • Impact of 2003 Workshop on book • Definitions and characterizations • Home grounds and the five critical dimensions • A risk-based approach to balancing agility and discipline • Observations

25. Summary of Home Grounds

26. Five Critical Decision Factors • Represent five dimensions • Size, Criticality, Dynamism, Personnel, Culture

27. Outline • Background • Impact of 2003 Workshop on book • Definitions and characterizations • Home grounds and the five critical dimensions • A risk-based approach to balancing agility and discipline • Observations

28. Using Risk to Balance Discipline and Agility - Overview

29. Risks Used in Risk Comparison • Environmental risks • Technology uncertainties • Many stakeholders • Complex system-of-systems • Legacy compatibility • Agility risks • Scalability • Use of simple design • Personnel turnover • Too-frequent releases • Not enough agile-capable people • Plan-driven risks • Rapid change • Need for rapid results • Emergent requirements • Not enough discipline-capable people

30. Three Examples • Agent-based systems • Small – Event Managers application • Medium – SupplyChain.com application • Large – National Information System for Crisis Management (NISCM) application • Each example results in a development strategy based on risk analyses

31. Event Managers Project • Small startup company • Diverse set of smaller agent-based planning applications • This project: support for managing the infrastructure and operations of conferences and conventions • Widening variety of options and interdependencies, makes an agent-based approach highly attractive

32. Event Managers Profile

33. Event Managers Strategy

34. SupplyChain.com Profile • Turnkey agent-based supply chain management systems • Distributed, multi-organization teams of about 50 people • Parts of applications are relatively stable, while others are highly volatile • Architectures are driven by a few key COTS packages that are also continually evolving

35. SupplyChain.com Profile

36. SupplyChain.com Strategy

37. NISCM Profile • Broad coalition of government agencies and critical private-sector organizations • Support cross-agency and public-private sector coordination of crisis management activities • Adaptive mobile network, virtual collaboration, information assurance, and information integration technology • Private-sector system-of-systems integration contractor

38. NISCM Profile

39. NISCM Strategy

40. Risk Profiles Across Examples

41. Outline • Background • Impact of 2003 Workshop on book • Definitions and characterizations • Home grounds and the five critical dimensions • A risk-based approach to balancing agility and discipline • Observations

42. No silver bullet Clear agile and plan-driven home grounds Increased demands for agility, velocity, quality Balanced methods are emerging Build methods up vs. tailor down Methods are important; people more so Values Communications Expectations management Summary of Observations

43. No Silver Bullet • Brooks’ werewolf concerns • Complexity, conformity, changeability, invisibility • Agile methods • Handle changeability and invisibility • Miss on complexity and conformity • Plan-driven methods • Handle conformity and invisibility • Miss on changeability and complexity

44. Future Applications Need Both • Historically • Many small, non-critical, well-skilled, agile culture, rapidly evolving projects • Many large, critical, mixed-skill, ordered culture, stable projects • In the future • Large projects are no longer stable • Maintenance of extensive process and product plans will become too expensive • Complexity and conformity werewolves are waiting for agile projects • Attributes of both approaches will be needed

45. Balanced Methods are Emerging • Agile methods • Crystal Orange • DSDM • FDD • Lean Development • Plan-Driven methods • Rational Unified Process • CMMI • Hybrid • Boehm-Turner Risk-based • Code Science/Agile Plus

46. Build up – Don’t Tailor Down • Plan-driven methods traditionally • Define heavyweight process guidelines • Advocate tailoring • Treat mass of processes as security blanket • Agilists traditionally • Begin with the minimum • Add as needed (and justified by cost-benefit) • Start small; extend only where necessary

47. Maybe its not the methods! • Agile movement has echoed a long line of warning calls • Success of agile may be due as much to people factors as to technology • Valuing people over processes is the most important factor in the manifesto I know I saw something about that in the process somewhere…

48. People • Of the people, by the people, for the people • Separations of concerns is increasingly harmful • A few quotes… • The notion of “user” cannot be precisely defined, and therefore it has no place in computer science or software engineering.Dijkstra • Analysis and allocation of the system requirements is not the responsibility of the software engineering group, but it is a prerequisite for their work.The Capability Maturity Model for Software. • Software engineering is not project management.Tucker

49. Values • Reconciling values is a critical people-oriented task • Stakeholders value different things • Software engineering is usually value-neutral • Every requirement, object, test case, defect equally important • Process improvement and plan-driven methods are inwardly-focused • Aimed at productivity improvement • Not higher value to customer • Agilists attention to prioritization and negotiation are promising

50. Communications • Face it, most engineers can’t talk, much less write • Need to bridge the customer-developer communications gap • IKIWISI* reigns • Rapid change increases need for solid communication • Few sources of guidance • Alistair’s Agile Software Development is a good starting place *I’ll know It When I See It