Version Control at HAO /AIM

1. Version Control at HAO/AIM Ben Foster and Pete Schmitt December 3, 2010 • Brief history of version control in the AIM section. • What is SVN and how can it be used to improve our development cycle? • Current structure of the AIM repository. • Hands-on demonstration of basic use of SVN in the TIEGCM development environment.

2. HAO/AIM History with Source Code Version Control Systems: CVS 2003 SVN 2009 SVN 2010

3. HAO/AIM History with Source Code Version Control Systems: 90’s CVS 2003 SVN 2009 SVN 2010 Our use was limited and superficial $$TGCMROOT (not svn repository) tiegcm1-8 tiegcm1.92 tiegcm1.93 • Where is development code beyond v1.93? • How is it managed among developers? tiegcm2.0

4. HAO/AIM history with Source Code Version Control Systems: CVS 2003 SVN 2009 SVN 2010 modsrc Our use was limited and superficial time $$TGCMROOT (not the repository) Sci-1 modsrc Sci-3 tiegcm1-8 tiegcm1.92 modsrc Sci-2 tiegcm1.93 Where is the latest development code? tiegcm2.0

5. Schematic Structure of TIEGCM in the AIM Repository: tiegcm_amie branches: tiegcm import trunk tiegcm1.93 tiegcm1-8 tiegcm1.92 tags:

6. Improved Use of SVN at HAO/AIM: Gang tiegcm_amie branches: tiegcm import trunk Liying tiegcm1.92 tiegcm1.93 tiegcm1-8 tags: Wenbin CCMC • Development takes place on the trunk and branches. • The trunk (HEAD revision) should always be stable. • Tags are a validated release record, used by the Community.

7. Some basic SVN commands: • View revision log of the repository, most recent first: • svn log –v $SVN/tiegcm | less • Check out a revision (e.g. the trunk), from the repository: • svn checkout $SVN/tiegcm/trunk tiegcm_trunk • Or, a specific revision: svn co –r469 $SVN/tiegcm/trunk • Information about the svn working copy (cd to tiegcm_trunk): • svn info • Commit log of current svn working copy, most recent first: • svn log –v | less • Edit a source file, e.g., change crit(2) in cons.F • Check svn status, and show simple diffs of local changes: • svn status • svn diff • To “go back”, type svn revert, followed by svn status

8. Basic SVN commands, cont. • A week later, you receive email of a commit to the trunk (or you find out via svn stat -u). You can update at any time: • svn status –show-updates (same as svn stat –u) • svn diff –rHEAD • svn update • If the commit overlaps your local changes, SVN will signal a conflict. You must resolve all conflicts before the update will succeed.

9. Why SVN for TIEGCM Developers? • Allows developers to stay on common, dynamic code-lines (trunk or branch). • Avoids uncontrolled code divergence, and consequent pain and confusion. • Allows developers to track incremental changes, facilitating coordination of individual efforts. • Allows time-travel along development lines. WhatSVN does NOT do: • SVN does not know any physics of the upper atmosphere. • SVN is not a substitute for communication. We must continue to make (hopefully intelligent) decisions as a coherent group. • (e.g., weekly meetings, email notification of commits, analysis of common code lines, when to merge, when to release, etc)

10. Three levels of change management: • Low-level:A typo or benign bug is found in the code: • Commit to the trunk and notify the group to update. • More complex bug-fixes may require testing, validation, and group decision. • Medium-level:A modest set of changes that will be committed to the trunk when complete: • Make modifications in local svn working copy, with periodic updates to keep up with changes on the trunk. • When the new code is ready, and approved by the group, commit to the trunk, and notify the group to do updates. • High-level: A larger project that may take several weeks or months to complete, and could disrupt the trunk: • Create and work on an svn “project” branch, doing commits on the branch, and merges from the trunk as needed. • When approved by the group, merge to the trunk, commit, and notify.

11. HAO/AIM Repository Directory Structuresvnls –v $SVN $SVN • HAO: SVN = file:///home/tgcm/svn • CISL: SVN = svn+ssh://arc.hao.ucar.edu/home/tgcm/svn timegcm tiegcm tgcmproc_idl tgcmproc_f90 glbmean

12. TIEGCM in the HAO/AIM Repositorysvnls –v $SVN/tiegcm $SVN/tiegcm trunk Primary development line tags Releases (generally fixed) branches Project development (parallel with trunk)

13. TIEGCM in the HAO/AIM Repositorysvnls –v $SVN/tiegcm/[trunk,tags,branches] $SVN/tiegcm src tiegcm1-8 src To checkout the tiegcm trunk: svn checkout $SVN/tiegcm/trunk trunk scripts scripts tiegcm1.92 doc doc tiegcm1.93 svn co $SVN/tiegcm/tags/tiegcm1-8 tags tiegcm_amie Branches can serve different purposes branches tiegcm_newbuild

14. Procedure to set up and execute a default run of the tiegcm trunk code: • Set environment variables in C-shell init file (~/.cshrc) (see later slides in this presentation) • Make and go to empty working directory on large disk. • Check out a working copy of the trunk: • svn checkout $SVN/tiegcm/trunk tiegcm_trunk • Copy default job script appropriate to the platform: • cp tiegcm_trunk/scripts/tiegcm-linux.job . OR • cp tiegcm_trunk/scripts/tiegcm-ibm.job . • Build the model and execute a default run: • tiegcm-linux.job & on Linux machines, OR • bsub < tiegcm-ibm.jobon IBM/AIX machines with LSF

15. The End (Party-Time!) Thanks for your attention A few reference slides follow

16. Some useful SVN commands • svn –hSee available subcommands • svn helpSee available subcommands • svn -h update Get help on the the update command • svn checkout $SVN/timegcm/trunk Check out timegcm trunk • svn stat Status of current working directory (modified files) • svn stat –show-updates Also show updates available from the repository • svnls –v $SVN/tiegcm/tags Long listing of this dir in the repository • svn commit –m”logmsg” Commit changes to the repository • svn diffDiffs of uncommitted mods in cwdwrt the repository • svn diff -r BASE:HEAD Compare local working directory to latest in repos • svn log Show log messages from commits on the cwd • svn log –r0:HEAD Log messages from beginning to latest in repository • svn copy Copy working dirs and URL’s with all svn info • svn export $SVN/tiegcm/tags/tiegcm2.0 Save unversioned copy of the v2.0 tag • svn merge $SVN/tiegcm/trunk@465 $SVN/tiegcm/branches/tiegcm_newbuild SVN reference: http://svnbook.red-bean.com SVN at CISL: http://www.weg.ucar.edu/services/subversion

17. Some Related Links: • ViewVCfor AIM -- a web-based repository browser (CIT login required): http://aimdev.hao.ucar.edu/cgi-bin/viewvc.cgi • Default TIEGCM job script and namelist files: tiegcm-linux.jobtiegcm_default.inp • Instructions to setup unattended ssh, see: http://rcsg.rice.edu/rcsg/shared/passwordless_ssh.html http://www.cisl.ucar.edu/docs/access/internal/uft.html • SVN documentation: http://svnbook.red-bean.com http://www.weg.ucar.edu/services/subversion

18. Csh Environment Vars at HAO: setenvTGCMDATA /hao/aim1/tgcm/data setenvSVNfile:///home/tgcm/svn setenvSVNsvn+ssh://arc.hao.ucar.edu/home/tgcm/svn # # PGI and MPI compilers (pgf90v9.0) # ($PGI is set by /opt/local/etc/Cshrc) #setenv PROCESSOR `uname -p`if ( "$PROCESSOR" == "x86_64" ) then setenvPGI_PATH $PGI/linux86-64/9.0elsesetenvPGI_PATH $PGI/linux86/9.0endifsetenvMPI $PGI_PATH/mpi/mpichsetenv PATH $PGI_PATH/bin:$MPI/bin:$PATHsetenvLD_LIBRARY_PATH $PGI_PATH/lib:$MPI/lib:$LD_LIBRARY_PATHsetenvMANPATH $PGI_PATH/man:$PGI_PATH/man:$MANPATH

19. Possible Future Expansion of the AIM Repository • Pre-processing utility codes (f90, IDL, perl, etc): • Preparation of start-up history files at high resolutions • Creation and updates of data files for import to the models • TGCM specific utility commands (e.g., tgcmroot/bin) • A “data” branch of the repository: • Start-up history files for released model versions (netcdf) • Files used for data assimilation in the models • Validated model results, including history output, visualizations, control runs, analysis, etc. • TGCM website files, including download page