Our Journey to Cloud Cadence

2. Our Journey to Cloud Cadence thoughts to share from Microsoft Developer Division’s Agile Transformation Jeff Beehler Principal Program Manager, Visual Studio Cloud ServicesMicrosoft Corporation

3. Agenda • Experience of MS Developer Division • How we enable continuous feedback • A 21st century concensus

4. Global Distributed Software DevelopmentThe Visual Studio Team Common challenges Quality assurance Geographic distribution Holistic project view Collaboration and efficiency Flow of customer value

5. Developer Division size (according to TFS) 3,659 Active users last 7 days 681,004 Total checkins 426,443 Work items 16,855,771 Source code files 15 Terabytes of data Stats from DevDiv TFS Server…onedivisional server instance out of 42 throughout the company

6. DevDiv: Servant of Three Masters • Creates conflicting priorities among stakeholders • Should force a discussion of single backlog • At divisional level, reconciled largely through resourcing

7. VS2005 – Starting Culture Conway’s Law The best and brightest Autonomy, job rotation, promotion The currency of love Headcount Dysfunctional Tribalism Don’t ask, don’t tell Schedule chicken Metrics are for others Our tribe is better Our customers are different Waste Easy credit No interest charge for debt

8. Debt Crisis • VS 2005 Bug Step @Beta 1 • Deferral a common technique (see top line) • Teams carry undone work (both functional and ‘ility) • Endgame hard to predict

9. Product Cycle Pre 2005 3-4 Years to Build It Service It (10 years) • M0: Plan and cost the release • QFEs and hot fixes • M1…M3: Develop the code • err…M3.1…M3.3 Recode • Beta1: Integrate and pray • Service packs • Beta2: Test like hell • RC (release candidate) 0..n: Final builds • RTM: Ship it! • Business Model: One-time License

10. Three Waves of Learning to VS2012 • First wave: address technical debt • Second wave: amplify flow of value • Third wave: reduce cycle time

11. Moving from 2005 to 2008 Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

12. Moving from 2005 to 2008 Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

13. Focus of MQ (Milestone for Quality) Get Clean Understand our Debt Test Automation Product Code Bugs Test Case Bugs Test Infrastructure Bugs Put our Debt on the Table Cost our Debt Eliminate the Debt Stay Clean Measure our Quality Quickly & Consistently Anytime, Anywhere Quality earlier in the cycle Make life easier

14. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

15. Product Backlog End-to-End Scenarios, Value Props, Experiences, Features Scenario Scenario In your business, do you need to…? Epic: What if you could…? Would you buy or upgrade? Value Proposition Value Proposition Value Proposition Value Proposition Theme: Let me show you how to…? Would you use it that way? Experience Experience Experience Experience Feature Feature Feature Feature Feature Feature Feature User story: As a…, I can…

16. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

17. Quality Gates Applied to Features

18. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

19. Changes to Pre- and Post- Game Build it (2 years) Service It (10 years) • Planning: Start and groom product backlog • Quarterly: Powertools and Feature Packs for current release • MQ: Improve our practices, get ready for the next version • M0: What are our goals for this release? What customer value do we deliver? • M1…M35-week sprints: Develop and test the code • CTP (customer technical preview): Targeted customer release to collect feedback on mainline scenarios • Business Transformation from Packaged Product to Subscription • Beta1: First broad customer visibility; collect feedback • Beta2: Validate recent changes with customers; collect feedback • RC (release candidate) 0..n: Final builds • RTM: Ship it!

20. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

21. Engineering Principles • Drive quality upstream • Increase efficiency by doing things right the first time • Don’t defer work • Bug tail not filled with hidden work • Keep main branch customer ready • Enables consistent delivery of high quality CTPs • Product not tattooed with incomplete or unstable code • Ready to ship when the time is right • Use Feature Branches for new feature development • One branch per feature • Only RI feature branch when quality gates are met • Feature Branch  PU Staging branch  Main • Automate testing • Automated testing at multiple levels • Tests run anywhere

22. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

23. Interim E2E Perf Scenario Drilldown

24. Common practices Get Clean, Stay Clean Product Backlog Defining Done Scheduling Engineering Principles Measurement & Hardening

25. Beta 1 planning review Initial planning spike • Beta 2 review Actual Feature Flow in VS2008 Additional tightening @ Beta 2 Tightened process rules @ Beta 1 reduce WIP

26. 2005 Debt vs. 2008 Debt • 15x Debt Reduction! • Minimal Deferral • 2x Schedule Improvement • High Predictability • Huge Customer Sat Rise VS 2005 Beta 1 Product Bugs only VS 2008 Beta 1 ALL bug debt

27. Three Waves of Learning: VS 2012 • First wave: address technical debt • Second wave: amplify flow of value • Third wave: reduce cycle time

28. Having removed debt, we could focus on flow of value Trustworthy Transparency Flow of Value Reduction of Waste Focus on the flow

29. VS2010 Aspirational Lean Scenarios • Production mismatches • Waiting for build setup • UI regressions • Performance regressions • Missed requirements or changes • No repro • Planning black box • Late surprises • Build breaks • Parallel development pain • Bewildering admin • Butterfly effects or legacy fear • Code & fix What if you could experience No More…

30. Wave 2 Realizations • Actions beget reactions, good and bad • Product ownership needs explicit agreement • Grooming product backlog cannot be skipped • Backlog need qualities of service • Celebrate successes, but don’t declare victory

31. VS 2010 Recognition Community votes for best product Industry analysts recognize Microsoft leadership GARTNER: Magic Quadrant for Application Life Cycle Management 5 June 2012 IDC: IT PPM Market Landscape December 2010 Evaluating VS2010 VOKE: Test Market Mover Array July 2010 Evaluating VS2010

32. The Third Wave of Learning: VS 2012 • First wave: address technical debt • Second wave: amplify flow of value • Third wave: reduce cycle time

33. Customers

34. The three questions they vote on

35. And the votes are visible and discussed

36. Real Users Matter

37. continues Improved cycle time

38. Post-VS2012 Value Delivery Highlights • Visual Studio 2012 Quarterly Update 1 Team Foundation Service Kanban phase 1 Cross-browser testing SharePoint emulator for unit testing SharePoint load testing Code coverage for manual testing Pause and resume long-running manual tests Code map IntelliTrace integration with System Center • Kanban support • Drag & drop sprint planning • 400 character paths on server • Build drops • Service released and is no longer “preview” • Send work items in email • Changeset summary view • Inline viewing and diffing for images • Account rename

39. Agenda • Experience of MS Developer Division • How we enable continuous feedback • A 21st century concensus

40. Continuous feedback Ideas Priorities PRODUCT BACKLOG Live site Code & Test Testing Sprint Monitor OPS BACKLOG

41. Ideas Ideas Priorities PRODUCT BACKLOG Live site Code & Test Testing Sprint Monitor OPS BACKLOG

42. Now everyone can storyboard ideas

43. Now make it continuous as a request from a PBI

44. And capture the results in the backlog

45. Priorities Ideas Priorities PRODUCT BACKLOG Live site Code & Test Testing Sprint Monitor OPS BACKLOG

46. Work visible

47. Iteration Backlog on a Taskboard

48. Code & Test Ideas Priorities PRODUCT BACKLOG Live site Code & Test Testing Sprint Monitor OPS BACKLOG

49. Continuous Feedback while Coding Test a little Coding Feedback Loop Code a little

50. Testing Ideas Priorities PRODUCT BACKLOG Live site Code & Test Testing Sprint Monitor OPS BACKLOG