1 / 31

Integration of Version Control and Code Migration Processes

Integration of Version Control and Code Migration Processes. JA-SIG - December 7, 2004 - New Orleans, LA Jim Thompson Division of Information Technology University of Wisconsin – Madison https://mywebspace.wisc.edu/jfthomps/vc. University of Wisconsin - Madison.

emmly
Download Presentation

Integration of Version Control and Code Migration Processes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Integration of Version Control and Code Migration Processes JA-SIG - December 7, 2004 - New Orleans, LA Jim Thompson Division of Information Technology University of Wisconsin – Madison https://mywebspace.wisc.edu/jfthomps/vc

  2. University of Wisconsin - Madison • public land-grant university (est. 1848) • home of the Badgers • Big Ten conference (11 with Penn State) • over 40,000 students • research expenditures • second in nation among public universities • third in nation among all universities • Madison is the capital of Wisconsin JA-SIG 12/7/2004

  3. My work group • part of Division of Information Technology • we design, build, acquire, install and support Internet infrastructure applications (a.k.a. upper middleware) • E-mail • Calendar • My WebSpace • Portal JA-SIG 12/7/2004

  4. Today’s presentation • Some background info about our portal • How we use CVS, tags and scripts • How tags and scripts work with our portal • Methods can work for other applications • Samples from a document that tells how • Additional tags for our new portal • Some plans for tracking conversion • Maybe some of this can work for you JA-SIG 12/7/2004

  5. My UW-Madison (our portal) • http://my.wisc.edu provides access to • E-mail • Calendar • My WebSpace • Enrollment • Financial Aid • Classes on the web (Learn@UW & WebCT) • Course Resources • Grades (entry and review) • Guest Accounts (demo Annie Student & Andi Advisor) JA-SIG 12/7/2004

  6. Some My UW-Madison statistics • September 2003 – over 1.3 million logins • Over a million logins/month thru May 2004 • September 2004 (new usage records) • Over 1.85 million logins • By 49,900+ users • September 2, 2004 (first day of instruction) • 85,739 logins by 29,676 users • About 6,000 users over lunch hour • Upped max Apache threads from 300 to 450 JA-SIG 12/7/2004

  7. My UW–Madison - some lingo • MUM = My UW–Madison • MUMMI = module inventory • MUMMR = migration request • MUMsters = developers • mumdev = development/test box (Solaris) • mumweb1,2 = web servers (Linux) • mumapp3,4 = app servers (Solaris) JA-SIG 12/7/2004

  8. MUM Service Goals • 24x7 availability (but not all connected systems are always up) • Handle peak traffic without slowing down • Good response time from called servers • Minimize outages (frequency & duration) • Weekly updates of production JA-SIG 12/7/2004

  9. Production Updates • All portal code is on our servers • Prefer weekly releases • Prepare a staged directory (Monday PM) • Deploy early Tuesday (between 5 & 7 AM) • Prefer “rolling restarts” to avoid outages JA-SIG 12/7/2004

  10. JA-SIG 12/7/2004

  11. The road to production • Files migrate thru 5 environments • Development • Test • QA • Production • Demo • 3,000+ files in each environment JA-SIG 12/7/2004

  12. Overview of environments • Development • the Wild West • CVS available, but not required • Test • Files must flow thru CVS to get to test • Developers apply “test” tag in CVS • Updates Mon-Fri 4:15 AM and “right now” • All test migrations stop/restart test JA-SIG 12/7/2004

  13. Overview of environments • QA • Files flow thru CVS • Developers request migration via MUMMR • Run load tests here • Production • Files flow thru CVS • Demo • Files flow thru CVS • Nearly as important as production JA-SIG 12/7/2004

  14. Files flow thru CVS • Question: How do we control the flow? • How to mark a file for promotion? • How to know what files got promoted… • Which version? • Where? • When? • Answer: CVS tags • Drive the process • Indicate what’s where, since when JA-SIG 12/7/2004

  15. Some CVS tags JA-SIG 12/7/2004

  16. CVS tag maintenance • Who maintains which tags? • Developers apply and move test tags • MUM admins apply qa, prod, demo tags • Scripts apply and move in and outa tags • Scripts create history tags • What’s the purpose of the outa tags? • Track deletes as well as adds and changes • Encourage developers to clean up • Avoid permanent loss of old code (in Attic) JA-SIG 12/7/2004

  17. JA-SIG 12/7/2004

  18. Why a MUMMR? • Reproducible instructions • QA is where we test the MUMMR • Complete list of files? • Clear instructions? • If we find problems, fix and resubmit MUMMR • Increase odds of successful production move • Documentation • Tell Help Desk and others what’s coming JA-SIG 12/7/2004

  19. Files flow JA-SIG 12/7/2004

  20. CVS tags - Who cares? • Developers • Where’s my code? • Project Managers • Which version was installed where and when? • Migration Controllers • What’s scheduled to move? • Everyone • What’s the difference between X and Y? JA-SIG 12/7/2004

  21. How comparing tags can help • Preparing to update 100+ production files • Apply prod tags according to MUMMR • Refresh export of files tagged prod • Compare inqa and prod • If all files are moving, there should be no diffs • All differences should be examined • Partial migrations are opportunities for problems • Developers can use same technique • Compare intest and inqa to get MUMMR right JA-SIG 12/7/2004

  22. That’s the main story, but… • Environmental files • permanently branched in CVS (never merged) • 16 branches for properties.txt • convert: Tomcat to WebSphere to WebLogic • sometimes had 8+ active environments • Sometimes had multi-flavored regular tags • wastest, testwls, wasprod, prodwls, etc. • always kept test tag for current • normalized tags after conversion to WebLogic JA-SIG 12/7/2004

  23. Some environmental branches • Current • my-dev35-wl-mumdev • my-test35-wl-mumdev • my-qa35-wl-qa • my-prod35-wl-prod • my-demo35-wl-prod • Former • bob-marley (ran on a PC) • my-spheredev-mumdev JA-SIG 12/7/2004

  24. Controls in CVS • Regular application files • tag changes limited via CVS filter (taginfo) • Environmental files • anyone can grab a branch • commits trigger e-mail notice to MUM admins • commits on dev or test branches are OK • commits on other branches, if not by MUM admins, may cause questions JA-SIG 12/7/2004

  25. Using this stuff elsewhere • We built a similar system for a COBOL app • CVS repository and app on AIX • 5 files in production app directory • Clean A-B-C design yields easy scripts • A-B-C = separation of App, Bin, Config files • Contents of document aimed at 3 audiences • Managers (General Overview ~ 17 pages) • Designers (Design Decisions ~ 5 pages) • Implementers (Details & Example Scripts ~ 30 pages) JA-SIG 12/7/2004

  26. Samples from our document • Our CVS taginfo filter calls tagcheck.pl • tagcheck.pl lets admins change any tag • How do we know tagcheck.pl is working? • see S.220 for test plan • brief primer on basics of CVS - G.040 • blank MUMMR – X.110 • http://hubcap.doit.wisc.edu/~jft/intvc JA-SIG 12/7/2004

  27. Moving to uPortal • What will change and what won’t? • Good bye, Makefiles… Hello, ant • Basic CVS tags and processes will continue • Change from shell scripts to build.xml • Call cvs-logntag.pl from build.xml • Some new tags for uPortal framework • Some tags to help track conversion JA-SIG 12/7/2004

  28. CVS tags for uPortal framework • Keep vendor files on vendor branch • Separate vendor branch for each vendor • Keep our local modifications on trunk • CVS tags to identify • Each vendor release • Our internal testing of each release • Our internal release for each environment • https://mywebspace.wisc.edu/jfthomps/vc/CVS_up.html JA-SIG 12/7/2004

  29. Tracking uPortal conversion • Conversion will take months • Changes to current portal not likely to stop • How to determine conversion status • Modules (probably via MUMMI) • Files (via anecdotal CVS “up” tags in old repo) • https://mywebspace.wisc.edu/jfthomps/vc/Tracking_conversion_progress.html • Reports posted on web each Wednesday JA-SIG 12/7/2004

  30. Can this work other places? • It depends… • Management commitment • Compatible design (A-B-C) - see D.030 • Flow control (adjust as you wish) • Cautions D.080S.250-260 and checks S.350 • We’ve seen optional Version Control • See M.030 • If side doors are open, they will be used JA-SIG 12/7/2004

  31. Questions ??? “Prediction is difficult, especially about the future” – Yogi Berra JA-SIG 12/7/2004

More Related