Release Management/SCM with TFS Scott Colestock

1. Release Management/SCMwith TFSScott Colestock Presented by

2. About me - • Consulting independently in the BizTalk 2004/2006 space for about 3 years • Blog: www.traceofthought.net • Recently focused on VSTS/TFS (speaking/writing/production rollouts)

3. Agenda • Branching & Merging • Mechanics within TFS • Strategies • How does Team Build help with SCM? • Carrying through to physical environments • TFS Deployer

4. Software Configuration Management • The process of identifying and selecting all the elements of a software system that create the correct and cohesive set - at defined points in time • Not unlike managing bill-of-materials for a highly complex manufactured product • Large topic – we’re focused (primarily) on branching & merging strategies… • Poor SCM leads to development, testing, and operational inefficiencies

5. Branching Mechanics in TFS • TFS implements “path space branching” (vs. “version space”) • Similar to subversion • CVS/others hide the branch… • Branching in TFS is essentially like doing a “deep copy” of a folder on your local drive • But branch “parent” is preserved (where in the tree you branched from) • Originating changeset is preserved • Ability to selectively merge parent changesets into the branch

6. Branching Mechanics in TFS • Path space branching means you see the branch folder directly in the Source Control Explorer – easy to navigate, retrieve, and establish permissions per branch • TFS implements an immediate branching model, rather than deferred (i.e. branch-on-change) • More meta-data created at the point of branch operation (though just one copy of underlying files until changed in branch) • A bit more useable – contents of branch are clear

7. Branching Mechanics • When creating a project, plan for future branches by placing initial check-in under a “Dev” or “Main” folder – not at your Team Project’s source root • Branch operation asks you to specify a “branch target” – a location in the source control hierarchy where the branch should live

8. Branching Mechanics • Most of the time, this will be a peer – e.g. $/OrderSystem/QA (or whatever the purpose of the branch is) • Keep clear on the differences between branch parent/child relationships and the physical structure you observe on disk and in source control repository…

9. Branching Mechanics • Branch operations result in a set of pending changes (of type “branch”) which must be checked in before branch is made real • Branching can cross TFS projects as well • Useful for sharing code across projects when you still desire to build/unit-test independently

10. Branching Thoughts • Mental model: Just like workspaces provide isolation for a given developer, branches provide isolation for a group of developers • Only branch if you have to, and at the point in time required • Branches can happen “after the fact” via branch-from-label, branch-from-changeset, etc. • (Like at the point in time you need to create a hotfix…)

11. Branching Thoughts • Always ask: Will the cost of merging conflicts in “real time” be higher or lower than the cost of merging conflicts between branches? • Isolation can offer greater near-term productivity (more parallelism) • Trade off is the risk of instability introduced by the merge (which increases with time-between-merge) • Merges can often need to flow both ways (“reverse” and “forward” integration) • Don’t use branching to maintain different configurations of the same product. Do this at build time, packaging time, or deployment time, etc.

12. Simplest Branching Model: None • Keep in mind: • TFS Team Builds are labeled automatically, or you can apply your own label • You can branch-by-label at a later point in time if you need to… • You can modify the contents of a label for small-scale changes • If your team moves forward together and you have a single release point (e.g. your web site) you may never need to branch • Do you need to actually do service releases/patches? • Or do you just do another “point” release each month or so? • Do people work “ahead” on items you don’t intend to release in your next drop?

13. (Label Detour)

14. Branch by Label

15. Label Thoughts • Labels allow you to apply a marker to the current (or past) state of a file or folder • Usually used to denote a milestone or release • You can apply multiple labels to a version of a file or folder… • Contents of a label can be edited after the fact (but no version history of the label is maintained) • Team Build automatically assigns a label to the set of files associated with each build that it creates. • Trivia: ‘Label scope’ is a path within source control - underneath which no other labels with the same name can be created • When you use Source Control Explorer, scope = project root

16. Visualizing Labels • The “contents” of a label is a set of changesets • The minimum number of changesets needed to describe the state • All that matters is the “aerial view” • The fact that other changesets were involved in reaching the labeled state is irrelevant • Label can be described by the content of the changesets “visible” from the top looking down… Label: 1.05 Release

17. Label Mechanics • In Source Control Explorer, right-click at appropriate level and select “Label” to apply a label • To “latest” • Or by date • Or by changeset • (Or by label – Build “Dev Debug_20070319” is also Version 2.4) • To find/delete/edit – use File-Source Control-Label-Find Label • Edit allows you to choose additional “participants” in the label or later versions of existing participants • Use http://shurl.org/tfskicks for advanced label features • Label comparison • Label changeset contents (union of all participant changesets) • Label work items

18. Branching Models at 10,000 ft • Lots of religious debate! But many would say… • Develop most new code on the “trunk” • Aka “main line”, aka “development branch” • Branch per feature and/or per team if absolutely required • Branch to stabilize and release • Feature Branch • Isolate work on new or experimental features that might cause instability to the rest of the project. • Isolate work on interface changes that will cause instability for the rest of the project.

19. Branching Models at 10,000 ft • Team Branch • Isolate sub teams so they can work without being subject to breaking changes from other teams. • Isolate sub teams so they can work towards unique milestones. Team branches often have feature branches beneath them. • Release Branch • Branch for builds that will require you to perform ongoing service/maintenance (sub-branch for hotfixes) • Branch to pursue/stabilize multiple releases in parallel

20. Considerations in Branching • Consider structuring branch trees so you only need to merge up/down the branch hierarchy – not across • Avoids need for “baseless merge” (tf /merge baseless …) • Remember that branch hierarchy likely doesn’t match repository and file system layout in terms of the “tree” • Merging a change back “up” to main line and “down” to another branch requires care • If you branch too deeply (Main -> Team -> Feature -> Experimental work…) you will introduce latency • Merges require conflict resolution – and bad merge decisions can happen, so the resulting build must be tested thoroughly

21. A few branching use cases…

22. Branching to aid nightly build stability… • Dev (aka trunk/main) branch for active check-ins • “It compiled locally and on build server” • All developers have read/write • Continuous builds (CI) should happen here • Use for quick feedback & feature tests • Integration branch for integration • “It passed build verification tests” • All developers have read; limited people have read/write • Only merge non-breaking changes from “Dev” • Use as basis for perf, security, integration tests (might call “Test” branch for this reason) • Nightly builds • Can use as basis for production branches

23. Branching to stabilize new feature • Dev branch may have child branches for each feature requiring active, overlapping & isolated development • Dev • Feature 1 • Feature 2 • Feature 3 • Only “worth it” if your features frequently collide (broken builds and/or frequent conflicts) and… • the cost of merging conflicts in real time is higher than the cost of merging conflicts between branches!

24. Branching for independent teams… • Dev branch may have child branches for each team that requires active, overlapping & isolated development • Dev • Team 1 • Team 2 • Team 3 • Only “worth it” if your teams frequently collide (broken builds and/or frequent conflicts) and… • the cost of merging conflicts in real time is higher than the cost of merging conflicts between branches! • Sometimes true for geographically distributed teams where communication overhead is high

25. Branching to isolate changes in external dependencies… • “Dev” branch may have a child branch where changes to third party libraries (or internal frameworks) are absorbed & tested to ensure no breaking changes • Merge to Dev/Main can occur once testing complete

26. Branching to support a release • Dev branch – everyone works here if team marching together… • Integration/Test – merge from Dev and to Production from here (always the basis for production branch – not last release) • Production branches - • “Release 1” • Hot fixes are likely labels on the release branch, not new branches… • “Release 2” • If different teams are pursuing work on different releases and they require isolation, team may work directly in release branches to pursue stabilization or service/maintenance work • Fixes can be merged back to Dev (aka trunk) • But carefully: The fix might not apply directly! • V.next work can happen in Dev branch once the release branch has been created. Can be shelved temporarily until that happens.

27. Branching to support a release • Branch models like this are termed “Mainline development” or “branch by purpose” • Note: Fixing a bug in Release 1 that also exists in Release 2 requires careful thought • Can fix at earliest point in time (say Release 1), merge back to trunk, then merge forward to Release 2 • But this includes risk of taking unwanted main line change into Release 2 • May wish to attempt baseless merge instead in some circumstances

28. Alternate branching method • Challenges to using each release branch as basis for next release (as opposed to using mainline): • Not intuitive for developers to point their environment to different branches (though vs.net makes this easier) • Difficult to work on multiple releases at once – even more difficult to work on future release • Private branch will require baseless merge • Easier to work in one branch, knowing work is promoted to appropriate branch • Advantage is that bug fix in old release only has to be propagated forward (not up to trunk & back down, or across) • Lots of debate on which model is better…

29. Branching to model promotions… • Branch per environment – Dev, QA, Stage, Production… • Labels mark release points – branch-by-label later if you need to for special purpose • This method generally accompanied by fine-grained changesets with clear work-item associations • As you merge from Dev to QA (etc.) you can hand-pick changesets • Or by work item using TeamCM

30. $/…/Dev $/…/Test $/…/Prod 26 25 24 21 14 27 18 13 23 15 12 17 19 9 26 15 10 27 24 18 13 16 17 9 22 20 25 23 Team Foundation Promotion Modeling (Branches of varying quality) foo.cs bar.cs foo.cs After 12: Branch to test = 13After 16: Merge latest to test = 17After 17: Branch to prod = 18After 22: Merge 21, 22 to test = 23After 23: Merge latest to prod = 24After 25: Merge latest to test=26After 26: Merge latest to prod=27 bar.cs foo.cs bar.cs

31. Merging Mechanics in TFS • When you merge, you initiate from the “source” – where the changes are coming “from” • Then choose merge “target” • Can merge by: • Latest version • Label • Date • Changeset

32. Merging • Can merge by: • Selecting individual changeset(s)… • Useful for handpicking bug fixes or changes to merge • Closed bugs (work items) will tell you which changesets are associated fix/change • Must be a contiguous range of changes for a given merge operation (ouch) • Keep your changesets fine-grained ! • Or use TeamCM to merge by work item

33. Merging • Merge includes content additions, deletions, un-deletions, renames, moves…not just changes between two files! • Merge operations can require conflict resolution if development is occurring in the merge target • Note that merge requires your workspace to have “visibility” to all branches involved • Note that merge process itself doesn’t update the target branch on your local disk – but that is where merge will take place from !

34. Merging • Merge operations result in a set of pending changes (of type “merge”) which must be checked in (and will result in single changeset) • Merge history maintained for you – system knows what the “remaining” merge candidates are from branch parent to branch child (unlike svn)

35. Agenda • Branching & Merging • Mechanics within TFS • Strategies • How does Team Build help with SCM? • Carrying through to physical environments • TFS Deployer

36. SCM Mileage from Team Build • Team Build can be used in “branch aware” fashion (i.e. a build type per branch) • Multiple build types can be created for each branch… • In Debug, Release variants… • For continuous integration, nightly schedule, or on-demand • With different levels of post-build automated testing • Each producing a list of changesets that have been incorporated since last build (though merge operations will appear as one changeset if checked in together) • Depending on which CI solution you use, you can enact dependency replication as part of build • TFS Integrator will do this… • (Or use MSDN sample)

37. SCM Mileage from Team Build • If you have a team build type per branch… • And you have continuous integration configured for each branch… • Then every merge between branches (Dev->Test or FeatureX->Main) will trigger a build • The “branch build” can involve unit tests that are assessing (in part) whether the merge operation produced good results • Valuable, since merges often error prone

38. CI for TFS 2005

39. CI for TFS 2008

40. Agenda • Branching & Merging • Mechanics within TFS • Strategies • How does Team Build help with SCM? • Carrying through to physical environments • TFS Deployer

41. SCM Definition Revisited • “The process of identifying and selecting all the elements of a software system that create the correct and cohesive set - at defined points in time.” • Once you have a branching/merging strategy in place, and a corresponding set of builds that occur based on those branches… • Carry it through to a set of automated deployments to corresponding physical environments!

42. TFS Deployer (Readify) • See http://shurl.org/tfsdeployer • Installed as an agent on servers that you must deploy to • Dev environment, Test environment…Production? • Listens to build quality change events • Build quality is edited within build store in vs.net • Build store notifies event service, which lets all subscribers (including TFS Deployer) know about quality change

44. TFS Deployer (Readify) • TFS Deployer uses deployment mapping file (checked into subfolder of team build type) to determine what (if any) action to take • Deployment file might indicate that a deployment for test servers should be kicked off when build quality transitions from “Dev Accepted” to “Test Staging” • Deployment Mapping file associates servers to actions based on build quality transitions for a given build • Can deploy to a farm by having multiple servers listed • Actions are powershell scripts

46. Team Build and “Build Quality” • Build Quality names in Team Build are customizable – so choose to match your process! • Mitch Denny (Readify) suggests: • Unexamined • Development Staging/Accepted/Rejected • Test Staging/Accepted/Rejected • Production Staging/Accepted/Rejected • Transitions can be secured…

47. Synergies from the whole… • When you have a good branch/merge strategy… • With corresponding automated builds…(per branch) • And appropriate & integrated automated unit tests… • With automated deployments to corresponding physical environments… • You will find a whole host of positive team behaviors can emerge ! • Because time isn’t lost due to poor SCM (wrong software in the wrong place, incorrectly deployed) • Because the team can treat that infrastructure as a given… • From identified bug to a traceable, farm-deployed fix in 10 minutes ?

48. Extending release management to database artifacts: Visual Studio for Database Pros • Database Refactoring • Schema comparison • Data generation tools • Database unit testing tools • Database projects… • Ability to harvest all schema objects from existing scripts or live schema • Triggers, functions, security elements, sprocs, tables, view, constraints, indexes, keys, triggers, UDT… • House in a single project with both “script centric” and “schema centric” views

49. Visual Studio for Database Pros • Key idea is to bring database development work into the standard development lifecycle • Version control (branching/merging) • Collaboration with team • Unit tests • Ideal: Each team member has local copy of database, where changes to sprocs, etc. made in isolation – and then unit tested • Communication of schema/sproc changes to team and propagation through environments begins with check-in! • “Building” a database project is an operation that is relative to a target environment • And…it generates the correct update script for you • Customization (and pre/post scripts) available for cautious DBAs once you are targeting stage/production

50. Thanks -