80 likes | 195 Views
WLCG Software Lifecycle . First ideas for a post EMI approach. Basic Concepts. Use open, widely used process for integration EPEL repository and process as integration point https ://fedoraproject.org/wiki/Bodhi_Guide test repository for staged rollout
E N D
WLCG Software Lifecycle First ideas for a post EMI approach
Basic Concepts • Use open, widely used process for integration • EPEL repository and process as integration point • https://fedoraproject.org/wiki/Bodhi_Guide • test repository for staged rollout • using test component against production repository • WLCG repo for non EPEL compliant packages • Alternative paths for middleware clients (tarball, CERNVMfs) • EPEL derived • Product Teams operate independently • Software repositories during development • Build tools ( most use Fedora tools ( mock etc.)) • Test tools for unit and system tests • Fine grained bug and requirement tracking • Production readiness on scale can be only verified in production • Pilots and Managed Staged Rollout • As little central coordination as possible
What needs to be coordinated? • High level bug tracking (GGUS) • High level requirement tracking ( GGUS ) • requirement prioritization ( PreGDB twice a year, WLCG-MB ) - • Roadmaps • for requirement implementation • decommissioning of components, interfaces and APIs • move to new OS versions • Middleware Configuration ( TEG OPS WG5 ) • Middleware Deployment ( TEG OPS WG5 ) • this covers Pilots and Staged Rollout • Release announcement and documentation • who, what, when • Could be done via the WLCG-T1-Service Coordination Meeting • documented in an extended WLCGBaselineServicesTwiki page
Roadmaps for: • Requirement implementation • Part of Pre/GDB and WLCG-MB • Needs to be prepared well in advance • Decommissioning of components, interfaces, APIs • same as now: Pre/GDB discussions, MB decisions • Move to new OS versions • same as now: Pre/GDB discussions, MB decisions • Currently this is done independently for ARC, gLiteand the OSG middleware stack • maybe this could be coordinated more closely
Middleware Configuration • Complex problem • small sites profit from a simple tool • YAIM or RPM post install scripts • large sites use configuration management tools • Quattor, Puppet, cfengine ..... no generic support possible • need highly specific configuration • Less configuration would be great.... • but is not likely to happen overnight.... • Probably a mixture between a simple tool and improved generic configuration guide is needed • Working Group???
Middleware Deployment • Based on EPEL + WLCG repository • YUM as a tool • Clients also provided in a form for distribution with the application software (Application Area, tarball,..) • this is only the case for gLite clients, ARC and OSG managed independently • Pilot services are negotiated between Product Teams, Sites and Experiments ( no central coordination(needed) ) • Staged Rollout needs to be managed • coverage of experiment/service/major site config. • EMI WN is a good example • Site and experiment participation is crucial • can’t rely on ad hoc agreements, needs formal commitment • needs follow-up and coordination • Could be part of the mandate of an WLCG Operations Team • Small WG needed to agree on the details
Some Detailed Questions • Rollback with a roll forward repository? • For RPM based distributions via RPM Epoch • V1.2 is in production (epoch 0) • V1.3 released and generates an issue • V1.2 re-released with epoch set to 1 • For RPM V1.2 epoch 1 is then “newer” than 1.3 • Non-backward compatible changes? • Introduced in parallel along production versions • Like: SRM, GLUE • Support for legacy service/API is retired as any middleware component
General Questions • Is there a need for a common approach for staged rollout for all middleware?