1 / 18

Condor & Middleware: NMI & VDT

Condor & Middleware: NMI & VDT. Condor Team Experience. Condor works on many platforms Condor build & packaging experience Condor deployed in many environments. Condor: platforms. This isn’t new (Condor 6.0, 1999) Linux Intel Solaris (2 versions) HPUX IRIX (3 versions) Digital Unix

pmazzone
Download Presentation

Condor & Middleware: NMI & VDT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Condor & Middleware:NMI & VDT

  2. Condor Team Experience • Condor works on many platforms • Condor build & packaging experience • Condor deployed in many environments

  3. Condor: platforms • This isn’t new (Condor 6.0, 1999) • Linux Intel • Solaris (2 versions) • HPUX • IRIX (3 versions) • Digital Unix • We have experience with many platforms • Condor Today (6.6) • Linux Intel (3 versions) • Linux Alpha • Windows • Mac OS X • Solaris (3 versions) • HPUX • IRIX • Digital Unix • Tru64 • AIX

  4. Condor: builds & tests • We believe in nightly builds & tests • Running since early 2000 • Runs on every platform • Developers immediately know when something breaks

  5. Condor: build pool (subset)

  6. Condor: Many environments • Maintain large pool in CS department • Three platforms • Many diverse applications • Have assisted with many pools world-wide: • You’ve seen many of them today • Commercial • Research • Large & Small

  7. Question • Can we apply this expertise to building, packaging, and distributing Grid software? • YES • NSF Middleware Initiative (NMI) • Virtual Data Toolkit (VDT)

  8. What is NMI? • NMI is a software infrastructure for middleware, especially grid • Many groups contribute software to NMI • Building, packaging, and testing is based here in Madison, with Condor Team members

  9. 18 distinct platforms Builds distributed with Condor NMI Builds Sources (CVS) Build & Test Condor pool (30+ computers) Patching … Build Binaries Web GPT src bundles Test

  10. Builds benefit from Condor • Fault tolerance • Computer fails—build just runs later • Can use DAGMan for ordered builds • New platform? Just add a computer

  11. Condor & NMI • NMI infrastructure builds lots of software • Capable of doing nightly builds and tests • Condor is one of these pieces of software • Helps Condor with nightly builds and regular tests • We use Condor to build Condor (via NMI) We eat our own dog food

  12. NMI GRIDS Status • Globus • Condor-G • Network Weather Service • KX509 • GSI OpenSSH • MyProxy • MPICH-G2 • Grid Packaging Tools • Gridconfig • Gridsolve • PyGlobus • UberFTP • NMI Release 4: December 2003 • Six platforms: • RedHat 7.2, 8.0, 9.0 • Solaris 8 • IA64 RedHa • IA64 SuSE Linux • NMI Release 5: coming soon

  13. What is the VDT? • A collection of software • NMI software + extras tailored for specific collaborations • Common Grid middleware (Condor, Globus, and more…) • Virtual data software • Utilities • An easy installation & configuration mechanism • Goal: Push a button, everything you need to be a consumer or provider of Grid resources just works • Two methods: • Pacman: installs and configures it all • RPM: installs some of the software, no configuration • A support infrastructure • Coordinate bug fixing • Help desk • Understand community needs and wishes

  14. Condor Group Condor/Condor-G DAGMan Fault Tolerant Shell ClassAds Globus Alliance Job submission (GRAM) Information service (MDS) Data transfer (GridFTP) Replica Location (RLS) EDG & LCG Make Gridmap Certificate Revocation List Updater Glue Schema/Info prov. What’s in the VDT? • ISI & UC • Chimera & Pegasus • NCSA • MyProxy • GSI OpenSSH • LBL • PyGlobus • Netlogger • Caltech • MonaLisa • VDT • VDT System Profiler • Configuration software • Others • KX509 (U. Mich.)

  15. Will use NMI processes soon VDT Builds NMI VDT Test Sources (CVS) Build Binaries Build & Test Condor pool (30+ computers) Pacman cache Package Patching … RPMs Build Binaries GPT src bundles Build Binaries Test Contributors (VDS, etc.)

  16. Recent Success: Grid2003 • VDT was deployed on 27 sites • VDT/Condor Team members contributed expertise and support • VDT provided access to 2000+ CPUs • Condor-G used by most users • Close collaboration promoted understanding

  17. Many VDT Customers • GriPhyN collaborators • US-CMS • US-Atlas • LIGO • SDSS • Particle Physics Data Grid • European Data Grid Project • Enabling Grids for e-Science in Europe Project • LHC Computing Grid Project • Alliance Grid Testbed (AGT)

  18. Questions?

More Related