1 / 15

Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC scholia@lbl

Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC scholia@lbl.gov NERSC User Group Meeting September 17, 2007. Introduction to Grids. Q: What is a Grid? A: A Grid is a system that: 1) coordinates resources that are not subject to centralized control …

stacey
Download Presentation

Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC scholia@lbl

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC scholia@lbl.gov NERSC User Group Meeting September 17, 2007

  2. Introduction to Grids Q: What is a Grid? A: A Grid is a system that: 1) coordinates resources that are not subject to centralized control … 2) … using standard, open, general-purpose protocols and interfaces 3) … to deliver nontrivial qualities of service. - Ian Foster, 2002, http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf NERSC User Group Meeting, September 17, 2007

  3. Introduction to Grids • Grid computing provides the ability to share and aggregate heterogeneous, distributed computational capabilities and deliver them as a service. • Uniform set of software interfaces to access non-uniform and physically distributed compute and storage resources. • Resources are part of a larger whole: • Compute Grid • Data Grid NERSC User Group Meeting, September 17, 2007

  4. Software Overview • Globus Toolkit 4.0.3 • GridFTP for data transfer across the Grid. • GRAM for remote job submission and batch access. • GSI and X509 certificates for security. • Using OSG Software Stack based on the Virtual Data Toolkit (VDT). NERSC User Group Meeting, September 17, 2007

  5. Core Grid Services at NERSC • GRAM Gatekeeper • Allows you to run / submit, monitor and query jobs on any NERSC machine remotely. • Interfaces with various backend batch systems (PBS, Loadleveler, SGE) while presenting a uniform interface to the user (Globus RSL). • GridFTP • Provides high performance data movement between NERSC and remote site / client system. • Supports striped, third party and parallel transfers. NERSC User Group Meeting, September 17, 2007

  6. Certificate Services • All Grid services at NERSC use X509 Grid certificates for authentication and access. • NERSC Users can acquire a grid certificate from the DOEGrids site (https://pki1.doegrids.org/ca/). • User must register their certificate with NERSC in NIM. • Once the certificate has been approved in NIM, user can access any of the NERSC Grid Services with this. NERSC User Group Meeting, September 17, 2007

  7. Certificate Services • MyProxy Server • Allows users to store their certificates in a secure fashion, and generate short term proxy certificates that can be accessed from anywhere. • Users do not have to leave long term certificates lying around on all their client systems. NERSC User Group Meeting, September 17, 2007

  8. System Deployment Cln HPSS GF PDSF GF Cln Davinci/NGF GF Cln Jacquard GF Cln Bassi GF Cln GK GK GK GK SGE PBS PBS LoadLev MPI Jobs GK GF Cln GridFTP Gatekeeper Globus Client NERSC User Group Meeting, September 17, 2007

  9. Grid Clients • Job Submission Tools • globus-job-submit, globus-job-query • Condor-G • GridFTP Clients • globus-url-copy • uberftp • Certificate Management Tools • grid-proxy-init • myproxy-init, myproxy-logon • grid-cert-info NERSC User Group Meeting, September 17, 2007

  10. Open Science Grid • Distributed computing infrastructure for large-scale scientific research, built and operated by a consortium of universities, national laboratories, scientific collaborations and software developers. • Researchers from many fields, including astrophysics, bioinformatics, computer science, medical imaging, nanotechnology and physics, use the OSG infrastructure to advance their research. NERSC User Group Meeting, September 17, 2007

  11. NERSC and the OSG • OSG users have access to compute/storage resources including NERSC. • NERSC resources are made available to OSG users via Grid Interfaces. • NERSC users may be able to access the OSG by making a request to the OSG Engagement team. NERSC User Group Meeting, September 17, 2007

  12. Coming Soon … • Grid Services on Franklin • SRM Access to NERSC Systems • NGF • HPSS • NERSC Online CA • Simplify the process of acquiring cert • User will be given a short lived cert based on LDAP password NERSC User Group Meeting, September 17, 2007

  13. Sample Users • STAR • NREL • Planck • OSG - WRF • OSG - LIGO • ESG • Incite 8 and 13 NERSC User Group Meeting, September 17, 2007

  14. What it means to YOU! • You can submit, manage and coordinate jobs across NERSC systems (and across other grid sites) using a common interface directly from a client machine. • Especially useful for managing workflows across multiple systems / sites. • Job control can be integrated into software running on your desktop. • The ability to move data across sites using high performance parallel transfers. NERSC User Group Meeting, September 17, 2007

  15. Example MyProxy Bassi DaVinci NGF Remote Web Server • Get Certificate • Submit Job to Bassi • GridFTP to HPSS • Do analysis on DaVinci • Move results to web server HPSS NERSC User Group Meeting, September 17, 2007

More Related