Configuration Management with ClearCase

1. Configuration Managementwith ClearCase 1. Basic concepts for usage 2. Basic concepts for administration 3. Setup of example environment 27.09.2006 - 28.09.2006 12.10.2006 - 13.10.2006

2. Configuration Managementwith ClearCaseBasic concepts for administration

3. ClearCase environment

4. VOBs appear in views as ordinary directory trees

5. ClearCase client server system Registry host(registers view, vobs, clients,...) License host(floating user lics) Backup registry host VOB host View host (optional WEB server) Client host Release host most often on same host host at different location !! avoid same host !! same host, different hosts,central view server host ClearCase installation NOT required

6. ClearCase network cleartool hostinfo -long

7. Planning first : naming conventions, storage locations, accounts, groups, ... create a VOB cleartool mkvob -tag /vobs/name -public /<storage>/vobstore/name.vbs cleartool mkvob -tag /vobs/priv /<storage>/vobstore/priv.vbs mount the VOB cleartool mount /vobs/name what is difference between public and private VOB ? create a view cleartool mkview -tag name /<storage>/viewstore/name.vws start the view cleartool setview name list existing processes serving VOBs and Views ps -ef | grep server list the /view directory (ls -la /view) list VOB and view storage directories Exercises:

8. albd (Atria Location Broker Daemon) runs on each ClearCase host, starts on ClearCase startup (root) the master server process (lic manager on license host) lockmgr (lock manager) runs on each VOB host, starts on ClearCase startup (root) grant access to VOBs for db_server and vobrpc_server processes db_server (database server) process VOB db transactions (started by albd on request) vobrpc_server (remote procedure call server) handle requests from view_server processes forward pool access to vob_server view_server (for each view on host of view storage) started by albd, remains active until endview / rmview vob_server (one for each VOB) ClearCase server processes

9. ClearCase host configuration /opt/rational/clearcase/... /usr/atria -> /opt/rational/clearcase /var/adm/rational/clearcase/... /var/adm/atria -> /var/adm/rational/clearcase /var/adm/rational/clearcase/rgy/... /var/adm/rational/clearcase/config/... /var/adm/rational/clearcase/schedule/...

10. Examine the /opt/rational/clearcase/... Directories etc, config subdirectories Examine the /var/adm/rational/clearcase/... Directories rgy, config, schedule, log subdirectories Discussion: How to protect /var/adm directories and files to permit other users than root to administer ClearCase ? Use a special group, e.g. ccadmin Include /var/adm/rational/clearcase in the backup of ClearCase !! , because ClearCase installation may overwrite your settings. Exercises: host configuration

11. VOB storage db database s file storage d derived objects c text cache ClearCase VOB parameter mount point access meta data VOB tag versions‘ containers access cache

12. View storage db database .s view privates config spec access filter ClearCase View parameter view reference access View tag internal admin checkouts, temporary data The most important part

13. Region 1 Region n Region 2 VOB tag VOB tag VOB tag View tag View tag View tag VOB tag VOB tag VOB tag View tag View tag View tag VOB tag VOB tag VOB tag View tag View tag View tag ClearCase registry View storage VOB storage VOB storage View storage

14. list existing regions cleartool lsreg create a new region cleartool mkregion <region name> why multiple regions ? register Vobs and Views in new region cleartool mktag -vob ... cleartool mktag -view ... multiple registries with their regions why / when multiple registries ? how to organize ClearCase environment with registry / regions ? Exercises: VOB and View parameter

15. ClearCase implements a multi level permission system. From “outer level” to “inner level” this are: VOB storage directory (the .identity directory = VOB groups) cleartool protectvob .... VOB element access (rwx modes on VOB elements) cleartool protect .... view protections (rwx of view storage directory) depends on umask settings at view creation time view access mode (read/write , read-only) cleartool chview .... ClearCase permission levels

16. ClearCase uses a hierarchy of users for individual command permissions: superuser (root) VOB owner element owner owner of meta data type object creator of version or derived object members of an element’s group anyone for example, the cleartool rmelem command can be executed by the element owner and anyone higher in the hierarchy ClearCase hierarchy of command permissions

17. Maintain VOB owner, groups change VOB owner, add, delete groupscleartool protectvob .... Maintain VOB elements change element’s owner, group, permissioncleartool protect .... change element’s typecleartool chtype .... Maintain other VOB objects like, types, branches rename a type object -> what really happens ?cleartool rename ..... Change owner, group of types, branches, cleartool protect ..... Combine all changes with the find command cleartool find . ... -exec ‘cleartool protect ...’ Exercises:

18. ClearCase VOB maintenance A VOB is like a ballon. VOB scrubber deletes unnecessary contents, but physical storage remains allocated - until physical size is reduced by reformatvob to the real size which again will grow on request periodically running by CC scheduler VOB physical storage VOB realcontents physical storage real contents

19. create a VOB cleartool mkvob -tag <name> <VOB storage> cleartool mkvob -tag <name> -host ... -hpath ... -gpath ... <access path> before deleting, moving a VOBALWAYS UNMOUNT it everywhere delete a VOB cleartool rmvob [-force] <VOB storage directory> NEVER use ‘rm -rf <vob storage>’ !!!!! But anyhow :-) in case of .... cleartool unregister -vob -uuid <vob uuid> cleartool rmtag -vob <vob tag>; and don’t forget to kill the processes move a VOB reformatvob -dump ...; cleartool unregister -vob ....; ct rmtag -vob reformatvob -load ....; cleartool register -vob ....; ct mktag -vob ... ClearCase VOB maintenance

20. create a View cleartool mkview -tag <name> <VOB storage> cleartool mkview -tag <name> -host ... -gpath ... <access path> before moving a Viewstop the view (cleartool endview -server ...) delete a View cleartool rmview [-force] <view storage directory> | -tag <view tag> NEVER use ‘rm -rf <view storage>’ !!!!! But anyhow :-) in case of .... cleartool unregister -view -uuid <vob uuid> cleartool rmtag -view <vob tag>; and don’t forget to kill the view process move a View cleartool endview ....; cleartool unregister -view ...; cleartool rmtag -view cleartool register -view ....; cleartool mktag -view ... ClearCase View maintenance

21. Always have in mind: except checked out data, a view is a temporary storage of data (often mis-used as permanent storage) cleartool lsview -properties [ -full | -age ] .... Never delete a view with checkouts !!! A view with checkouts has been deleted the VOB still references the checkouts ! - how to get rid of it ? => cleartool rmview -uuid <view uuid> where view uuid is retrieved by ‘cleartool describe -long vob:<VOB tag>’ a view can be moved even in case it has checkouts (same to VOBs) because checkouts are tracked by VOB/View uuids and NOT pathnames ClearCase View maintenance

22. VOB backup before backing up a VOB,LOCK it with command ‘cleartool lock vob:<VOB tag>’ !!!! To copy the VOB storage directory NEVER use copy commands, which modifies the access time !!!! this would disable a correct VOB scrubbing -> VOB size increases ! E.g. gtar --atime-preserve .... Steps for backup: cleartool lock vob:<VOB tag> cleartool unmount <VOB tag> kill -9 vob_server; kill -9 vobrpc_server; kill -9 db_server copy vob storage directory (before check for remote pools) cleartool mount <VOB tag> cleartool unlock vob:<VOB tag> ClearCase backup

23. VOB backup - another procedure using VOB snapshots reduces the time of the VOB lock a VOB has to be prepared for snapshot backup with command<CC install dir>/etc/vob_snapshot_setup -snap_to <dir> ... <VOB tab> it has to be scheduled in the CC schedulercommand is vob_snapshot it locks the VOB; copies the database (the db directory) to the snap_dir(but it uses tar - BAD); unlocks the VOB; afterwards the VOB pools are copied to the snap_dir make sure, that the snap_dir is backed up by normal backup procedures ..... ClearCase backup

24. VOB backup - another procedure using VOB snapshots reduces the time of the VOB lock a VOB has to be prepared for snapshot backup with command<CC install dir>/etc/vob_snapshot_setup -snap_to <dir> ... <VOB tab> it has to be scheduled in the CC schedulercommand is vob_snapshot it locks the VOB; copies the database (the db directory) to the snap_dir(but it uses tar - BAD); unlocks the VOB; afterwards the VOB pools are copied to the snap_dir make sure, that the snap_dir is backed up by normal backup procedures ..... The VOB snapshot backup is only listed for completness The author does NOT recommend to use it !! ClearCase backup

25. VOB restore of a complete VOB cleartool umount <VOB tag> on all hosts kill the vob_server, vobrpc_server, db_server processes save the current VOB storage to some place; after that DELETE original restore the VOB storage from backup to ist original location cleartool mount <VOB tag> on hosts where needed Hint: Try to establish backup via file system snapshot utilities, e.g. Veritas netbackup. ClearCase backup

26. VOB restore of only some elements you can NEVER have a copy of a VOB in the same registry therefore: use a different host with a different CC registry restore VOB from backup cleartool register -vob ...; cleartool mktag -vob ...; cleartool mount ... create a temporary VOB: cleartool mkvob -tag ....; relocate needed elements from restored VOB to temporary VOBcleartool relocate <source in restored VOB> <target in temporary VOB> cleartool umount <tmp VOB>; kill vob_server, .... copy temporary VOB storage to host of target registry cleartool register -vob <temporary VOB storage>; cleartool mktag .... Use again cleartool relocate from temporary VOB to original target VOB to restore requested elements to their original location ClearCase backup

27. View backup before you copy the view storage directory the view_server should be stopped to avoid further changes during the backupcleartool endview -server <view tag> use any backup utility to copy the view storage after backup start the view_server processcleartool startview <view tag> REMARK: if ClearCase is used “the right way” view backups shouldn’t be necessary. Only the importance of view private files determine the necessity of view backups. Discussion: What does this mean for checkouts ? ClearCase backup

28. View restore before you restore the view storage directory: stop the view: cleartool endview -server <view tag> save the current view storage to some place; after that DELETE original restore the view storage from backup start the view : cleartool startview <view tag> IMPORTANT: A restored view may not be in sync with the existing VOBs,e.g. regarding checkouts (references to/from VOBs) use ‘cleartool recoverview -tag <view tag>’ for database recovery use ‘cleartool recoverview -synchronize -tag <view tag>’ to synchronize the view with all VOBs, in which this view has created view private files ClearCase backup

29. Use the ClearCase backup registry server as your backup host ClearCase must be installed on this host have a subset of your ClearCase licenses assigned to your backup host in case of recovery: try to use hostname of original VOB host for your backup host -> re-registering of VOBs not necessary store the latest backup of your VOBs in the same structure as the VOBs are stored on the original VOB host(s) in case of desaster recovery - do NOT use view backups, but let users create new views ClearCase desaster recovery

30. The central schedule command cleartool schedule -get | -edit [ -schedule | -tasks | -acl ] A job is a task that has been scheduled to run one or more times with a set of arguments The scheduler is available on any host that runs the albd server The scheduler relies on two data repositories: a database of tasks available for scheduling /var/adm/rational/clearcase/scheduler/tasks/task_repository(a file editable with any editor) a database of jobs, or scheduled tasks /var/adm/rational/clearcase/scheduler/db(a db file only accessible via cleartool schedule) predefined tasks have task ids less 100 user defined tasks start with task id 100 The ClearCase scheduler

31. cleartool schedule -get -acl cleartool schedule -edit -acl cleartool schedule -get -schedule cleartool schedule -edit -schedule cleartool schedule -get -tasks cleartool schedule -edit -tasks the last two command invocations can be replaced byvi /var/adm/rational/clearcase/scheduler/tasks/task_registry how to integrate the CC backup in the scheduler ?? Exercises: ClearCase scheduler

32. Trigger types in ClearCase element trigger: attached to specific elements all element trigger: attached to ALL elements in a VOB, existing elements and newly created elements Triggers “fire” on ClearCase commands post operation: trigger fires after command execution pre operation: trigger fires before command execution, with exit != 0 it refuses command execution Triggers can be restricted by restriction list of types (element types, branch types, ...) by inclusion list of types (element types, branch types, ...) by excluded user list: trigger does not fire for specific users Trigger action => cleartool command or script ClearCase customization by triggers

33. Have a look to the manual page cleartool man mktrtype a pre checkin all element trigger type named PreCheckin cleartool mktrtype -all -element -pre checkin -nuser ccadmin -exec ‘exit 1’ PreCheckin what does it do ? Create your own trigger types and test them do not permit anybody to remove elements with command rmelem attach label IN_WORK to new version after checkin Two triggers recommended as standard in all VOBs remove_empty_branch avoid_evil_twin Exercises: ClearCase triggers

34. QUESTIONS&ANSWERS