1 / 18

TeraGrid Software Deployment

TeraGrid Software Deployment. JP Navarro TeraGrid GIG - Grid Infrastructure Group Software Integration. GIG SI staff currently working on software deployment Mike Showerman (SI Area Director) Charles Bacon, Jason Brechin, Lee Liming, JP Navarro GIG SI Partners

andra
Download Presentation

TeraGrid Software Deployment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TeraGrid Software Deployment JP Navarro TeraGrid GIG - Grid Infrastructure Group Software Integration TeraGrid Software Deployment

  2. GIG SI staff currently working on software deployment Mike Showerman (SI Area Director) Charles Bacon, Jason Brechin, Lee Liming, JP Navarro GIG SI Partners NMI/Grids: Peter Couvares, Becky Gietzel, Anatoly Karp Pacman: Saul Youssef Team Participants TeraGrid Software Deployment

  3. TeraGrid build & deploy History: software deployment during construction Scope: platforms, software components Future: goals motivating a new approach New build & deploy Approach overview Building software Software packaging Software configuration Software testing Inca Build time tests Initial Implementation Plan Globus 4.0.1 components, tgcp Outline TeraGrid Software Deployment

  4. TeraGrid Build & Deploy History Thru September 2004 • 6 build scripts: configure, build, package, install • Source -> installed software • Hand configuration • No uniform site customization method • No consistent pre-req, environment setup • Written by a volunteer network of s/w builders, admins • No quality standards • Not tested on every platform • Evolved TeraGrid Software Deployment

  5. Scope: Software 1 TeraGrid Software Deployment

  6. Scope: Software 2 • TeraGrid planning or likely to upgrade • Convergence possible TeraGrid Software Deployment

  7. Scope: Software 3 TeraGrid Software Deployment

  8. Scope: Software 4 TeraGrid Software Deployment

  9. Scope: Platforms TeraGrid Software Deployment

  10. Significantly lower the software deployment bar (effort, complexity) • GIG produced build methods that work on all TG platforms • GIG produced binary packages for all TG platforms • GIG supported installation and configuration methods • Timely stack evolution to meet project goals • Leverage NMI software build infrastructure to achieve: • Automated, consistent and reproducible software builds on all TG platforms • Tracked build inputs (source, dependencies, build methods) and outputs (binaries, logs) • Improve software quality and reliability • Improve consistency and interoperability between TG sites • Converge or interoperate with other Grids and Grid software • NMI, OSG, … • Retain the ability for Resource Providers “RP”, or anyone else, to build TG software from vendor source on their platforms Motivation for Change TeraGrid Software Deployment

  11. GIG SI Team (3+ FTEs) • Responsible for TG wide build & deploy • Delivers platforms specific binaries to RPs • Provides installation and configuration methods Leverage NMI software build infrastructure • On TG platforms • Launched from TG platforms • Separate build tools, and software source and binary repositories Leverage NMI software build scripts • Only possible for a subset of TG software The actual build scripts have to stand on their own • Usable without the build system to build by hand New Pacman based installation and configuration New Approach TeraGrid Software Deployment

  12. The Build System and TeraGrid • Build Facility for the TeraGrid • On demand build infrastructure, launched w/ Grid tools • (just another Grid application) • Launching and monitoring builds from TeraGrid • (just more Grid jobs) • Authenticated source extract and binary store happens on build machines (gridFTP) • Credential propagation with build jobs • Dependencies encoded in SoftEnv terms • Build time software dependencies • Run time software dependencies • Condor Hawkeye SoftEnv integration • Environment setup TeraGrid Software Deployment

  13. TeraGrid/NMI Build Grid grandcentral TG submit hosts TG platforms TG submit hosts TG platforms TG submit hosts TG platforms TG submit hosts TG platforms TG submit hosts TG build platforms TeraGrid Software Deployment

  14. TeraGrid CVS TeraGrid Software NMI Results DB Build Scripts Build Specs Sources Binaries Build Job Info TeraGrid/NMI Automated Build Architecture TG submit host grandcentral TG build host Condor Startd NMI Build/Test Framework TG Build Tools NMI Framework Wrapper Condor / Condor-G / DAGMan TG Software Build Scripts Existing NMI component Evolved TG component New TG component TeraGrid Software Deployment

  15. TeraGrid Software Binaries TeraGrid Software Deployment Architecture TG install host TG Install, Configure Tools TeraGrid CVS Pacman Caches Config Tools TeraGrid Software Deployment

  16. TG Install & Configure Tools Pacman 3.x • Common and simpler installation method • Dependency tracking • Verification • SoftEnv integration • Deinstallation Configuration Tools • Looking for good software configuration tools TeraGrid Software Deployment

  17. Software Verification • Inca • Availability verification • Version verification • User environment (SoftEnv) verification • Functionality checking (standalone & distributed) • Historical tracking • Grid Center build time testing (?) TeraGrid Software Deployment

  18. Initial Implementation Plan • Globus 4.0.1 striped gridFTP and RFT, tgcp 1.0 • Software in testing for about a month • Production in ~July with as many new methods as possible • Including dependent software • Globus 4.0.1 Pre-WS GRAM, RLS, MDS4 • July and beyond • WS GRAM • The rest of the evolving stack TeraGrid Software Deployment

More Related