1 / 22

Software Configuration Management and Integration Alberto Di Meglio EGEE JRA1 Integration Manager

Software Configuration Management and Integration Alberto Di Meglio EGEE JRA1 Integration Manager. EGEE Kick-Off Meeting – Cork – 18-22/04/2004. www.eu-egee.org. EGEE is a project funded by the European Union under contract IST-2003-508833. Contents. JRA1 Integration Mandate Team Objectives

aminia
Download Presentation

Software Configuration Management and Integration Alberto Di Meglio EGEE JRA1 Integration Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Configuration Managementand IntegrationAlberto Di MeglioEGEE JRA1 Integration Manager EGEE Kick-Off Meeting – Cork – 18-22/04/2004 www.eu-egee.org EGEE is a project funded by the European Union under contract IST-2003-508833

  2. Contents • JRA1 Integration Mandate • Team Objectives • The SCM Plan: an Overview • SCM and Integration Implementation • Configuration and Version Control • Build Systems • Release Process • Other Configuration and Change Control Procedures EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 2

  3. JRA1 Integration Mandate • EGEE JRA1 has a dedicated team for SCM and Integration activities • The vision of the team is that SCM and Integration must start before a single line of code is written and must provide an integrated development and collaboration environment • The mission is to provide a solid, reliable infrastructure to manage changes and release quality software based on industry-standard SCM procedures and tools throughout the entire duration of the project EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 3

  4. Integration Team Objectives • The Integration System must be simple, flexible and solid • Excessive diversity in tools, languages, technologies must be avoided • Centralization of processes and procedures is favoured • Whenever possible industry-standard tools and methods with a large user base and reliable, official support must be used • Making it work well on one platform is preferable to making it available on all platforms (at least initially) • Extension to multiple platforms and usage scenarios will naturally follow for a well designed, developed and integrated software • EGEE Middleware is a complex, multi-disciplinary, highly distributed project. This is part of the novelty and challenge of it and will require a novel approach to old and new problems and some courageous decisions EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 4

  5. SCM Plan Overview • A master Software Configuration Plan is being finalized now [1] • It contains basic principles and rules about the various areas of SCM and Integration (version control, release management, build systems, bug tracking, etc) • Compliant with internationally agreed standards (ISO 10007-2003 E [2], IEEE SCM Guidelines series [3]) • Most EGEE stakeholders have already been involved in the process to make sure everybody is aware of, contributes to and uses the plan • An EGEE JRA1 Developers’s Guide will follow shortly in collaboration with JRA2 (Quality Assurance) based on the SCM Plan • It is of paramount importance to deliver the plan and guide as early as possible in the project lifetime EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 5

  6. SCM and Integration Implementation • As soon as the SCM Plan is finalized, we will start the implementation phase (some steps already taken) • All tools, servers, procedures and communication channels will be put in place • Developer’s and User’s Guides will be produced with clear instructions about how to use and interact with the project and manage changes in the correct way • The complete infrastructure will be in place at the end of Month 3 (30 June) (MJRA1.1) • Goals for the first 12 months are to release incremental code baselines to test with a very rapid lifecycle (weekly) and finally release v. 1.0.0 (main milestone is MJRA1.5 at Month 10: integrated EGEE RC 1 released to testing) EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 6

  7. Version and Configuration Control • Based on CVS using CERN Central IT CVS service • Fixed directory structure for each module • Rules for tagging and branching (e.g. bug fix branches) • Common naming conventions for baseline and release tags • Configuration files to automate creation of workspaces • Used to enforce build reproducibility • Used to create private workspaces for developers • Authorization based on commitavail and tagavail according to SCM roles (developers, release managers, integrators, etc.) • Supported protocols: K4, SSH (pserver only for anonymous read-only). A CERN AFS account is required EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 7

  8. project src test doc config interface examples CVS Structure JRA1 EGEE CVS org.egee org.egee.subsys org.egee.subsys.comp1 org.egee.subsys.compN EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 8

  9. Workspaces • A Workspace is one of the basic SCM Patterns [4] • It is a private area where a build can be produced under configuration control • More independent workspaces can coexist on the same computer for parallel development • The workspace is created using a Configuration Specification File (CSF) • The CSFs contain all the information necessary to prepare the workspace extracting modules and other dependencies from CVS or the central software repository • There is a CSF file for the overall org.egee Middleware suite and one CSF for each subsystem • CSFs are stored in the corresponding modules in CVS and versioned so that any version of the MW suite and all its subsystems can be reproduced at any time EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 9

  10. Build System • Common Build System • A project-wide build system based on Ant to allow all developers, integrators and testers to manage private builds on their own computers with the same tools and settings (compilers, flags, configurations, etc) • Based on the concept of Private Workspaces, more than one independent workspace can be created on the same computer • Uses a set of common build and properties files to define targets and properties for all project modules • External dependencies are stored and versioned either in CVS or a central repository maintained by CERN SPI. All modules must use the SAME versions of the external dependencies across the project • Central Build System • Located at CERN and maintained by the JRA1 Integration Team • Based on the Common Build System, but automated by means of CruiseControl and Maven to generate a continuous, automated integration process and the overall project web site (Apache style). EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 10

  11. Build System (2) • The Ant build system provides the common build orchestration for all modules, not only Java • Modules can be built independently and use Ant or Make as specified by project requirements, independently from the module implementation language • A set of common targets has been defined for all modules (environment checks, style checks, compilation, unit tests, distribution, etc) • The entire MW suite, individual subsystems and individual modules can be built with the same set of targets • Quality Assurance checks are built into the build process and are mandatory. Failure to pass a quality check (coding styles, unit testing, minimum test coverage, documentation, etc) fails the build EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 11

  12. Build System (3) • Five main types of builds: • Continuous Incremental Builds: every few hours, mail messages are sent to developers who break the build • Nightly build: every night from clean snapshot • Weekly builds: every week from clean project configurations • Maintenance builds: used to fix software defects on existing releases • Private builds: whenever needed on the private development computers EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 12

  13. Build Tools • Ant: used for the general build management and all Java modules. The advantages/disadvantages of using it also for C/C++ and other languages modules are under investigation • Make + GNU Autotools: for C/C++ and other languages as necessary. Although this is the standard, familiar approach, it has a lot of drawbacks, especially for large, distributed projects (difficult to maintain and centralize, lots of external dependencies, poor internal dependency tracking, file based instead of task based, etc) • CruiseControl: used to automate the integration builds • Maven: used to automate the project web site, reports and documentation builds EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 13

  14. Packaging and Configuration • Installation packages for the EGEE Middleware will be produced by the integration team • Packages will be provided for the entire middleware and for all subsystems and deployment modules • Supported packaging formats are: • Source tarballs with build scripts • Native packages for supported platforms (e.g. RPMs for Red Hat Linux Enterprise 3.0, MSIs for Windows XP, etc) • Extracting and using modules directly from CVS is not recommended, since there is no guarantee the dependencies are satisfied outside the build system context • Configuration issues are being investigated with SA1 • The goal is to have a simple, common format for all configuration items, usable by standard installation/configuration tools • Some type of “Middleware Configuration Front-End” may also be provided in collaboration with SA1 EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 14

  15. Release Process • Developers works on code functionality and documentation and produce baseline candidates • Baseline candidates are continuously integrated and unit and smoke tested • A build is successful if all the functionality contained in the module is properly unit tested and documented. If a build is not successful, the baseline candidate is rejected and control returns to the developers • Baselines are automatically tagged if the build process is successful • When all baseline candidates in the application build list (Application CSF file) are successfully built, an application baseline is produced and passed to testing • If all test, certification and validation procedures are successful, the individual baselines in the build are considered eligible to be promoted to release if all required functionality is present • Development Release Managers promote eligible baselines to full releases • When all components have been released, functionality is frozen and the application is promoted to Release Candidate. • The Release Candidate is thoroughly tested, validated and certified, all critical and major bugs are fixed, most normal bugs are fixed. A number of Release Candidates are produced during a short number of iterations. • The final release is published EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 15

  16. Change Control Board • All public changes must go through a formal approval process • The CCB is tasked to collect and examine the change requests • Changes must be tracked and communicated as efficiently as possible • The role of the CCB (and SCM in general) is not to prevent change, but to organize it so it has only positive impacts on the project • It will meet periodically and on demand • Need to define composition of the board, very likely it will be composed of the JRA1 activity and cluster managers plus representatives of the architecture and design teams, deployment and applications • The relationship with the EGEE Architecture Oversight Committee (AOC) is under discussion EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 16

  17. Defect Tracking System • Bugzilla-like tool provided as part of the Savannah project portal • Provided and supported by SPI team at CERN • Need to add tools for collecting QA metrics (current reporting in Savannah is currently not sufficient) • Used also for change requests (for example API changes, external libraries version changes, etc). In this case, request are assigned to the Change Control Board for further evaluation EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 17

  18. Defect Tracking Cycle • Ten states: Open, Analyzed, Accepted, In Progress, Ready for Integration, Ready for Test, In Test, Approved, Ready for Review, Closed • Transition between two states subject to specific checks (Is it a defect? Can it be fixed? Test passed, Review passed?) • ITeam are first filter to assign bugs to people or categories if required EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 18

  19. Software Quality Assurance • Software Metrics are collected as part of the build process • Failure to pass a quality check fails the build • Additional checks are implemented in the version control system (coding style checks) • Software Defect and QA Metrics are collected from the defect tracking system • All reports are published every night on the project web site • Reports should be used by project and development managers to take corrective actions EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 19

  20. Collaboration Tools • Project Portal: Savannah from CERN SPI • Single entry point for the project, news, tools, documentation • Links to other relevant sites (CVS web browsing, builds status, package repository, project metrics and other artefacts) • Mailing lists from CERN or Terena • News, RSS feeds, etc, as necessary EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 20

  21. Contacts • ITeam mailing listproject-eu-egee-middleware-integration@cern.chList of the ITeam members and other activities representatives (closed list) • Integration supportproject-egee-middleware-integration-support@cern.chPublic support list for reporting integration and build problems EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 21

  22. References • [1] EGEE Middleware SCM Plan (https://edms.cern.ch/document/446241/0.6) • [2] ISO 10007:2003 - Quality management systems -- Guidelines for configuration management, ISO, 2003 • [3] http://standards.ieee.org/reading/ieee/std/se/828-1998.pdf (requires a subscription) • [4] Software Configuration Management Patterns, S.P. Berczuk, Software Patterns Series, Addison-Wesley, 2002 EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 22

More Related