1 / 27

Computational Grids and the Globus Project

Computational Grids and the Globus Project Ian Foster Argonne National Laboratory The University of Chicago Globus Co-PI: Carl Kesselman, USC/ISI Credits Thanks to the Globus teams at ANL, USC/ISI, NCSA, and elsewhere

andrew
Download Presentation

Computational Grids and the Globus Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computational Gridsand the Globus Project Ian Foster Argonne National Laboratory The University of Chicago Globus Co-PI: Carl Kesselman, USC/ISI

  2. Credits • Thanks to the Globus teams at ANL, USC/ISI, NCSA, and elsewhere • Steve Tuecke, Gregor von Laszewski, Steve Fitzgerald, Randy Butler, Brian Toonen, and many others • Research supported by DARPA, DOE, NSF, NASA; equipment from Cisco Systems

  3. Overview • Computational grids • Why, what, where • Globus project and its research activities • Focus on resource management • U.S. Grid deployment activities • NASA, DOE, DISCOM • Application overviews • Focus on on-line instrumentation • Summary

  4. nationally ... or internationally ... Context “Imagine a computing & information infrastructure designed to power research & education by providing pervasive access to all significant information resources”

  5. GridApplicationExamples Online analysis of instrument data Teleimmersion/distance collaboration TransAtlantic remote visualization/steering Record-setting distributed supercomputing Parameter studies with deadline scheduling

  6. “The Grid” • Pervasive, reliable, secure, high-performance access to leading-edge computational, data, instrument, display & human resources • New problem-solving methodologies based on this coupling of geographically distributed resources • Advanced network services that enable of the above (“middleware”)

  7. What are “Grids”? • Grid infrastructures support applications that • Involve coordinated use of end-system resources, often in different admin domains • Often require high performance • Require enhanced services beyond those provided by today’s Internet • Major production infrastructures & testbeds being deployed at this moment • Grid R&D sponsored by DARPA Quorum, NSF PACIs, NASA IPG, NGI (DOE in particular)

  8. Creating a Usable Grid :Grid Services (“Middleware”) • Standard grid services that • Provide uniform, high-level access to a wide range of resources (including networks) • Address interdomain issues of security, policy, etc. • Permit application-level management and monitoring of end-to-end performance • Middleware-level and higher-level APIs and tools targeted at application programmers • Map between application and Grid

  9. Unifying Concept:An Integrated Grid Architecture … a rich variety of applications ... Applns Appln Toolkits Remote data toolkit Async. collab. toolkit Remote sensors toolkit Remote comp. toolkit Remote viz toolkit ... Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Grid Services Grid Fabric Archives, networks, computers, display devices, etc.; associated local services

  10. The Globus Project: Argonne, USC/ISI, NCSA, Aerospace, NASA Ames, LBNL, others • Basic research in grid-related technologies • Resource management, security, adaptation, etc. • Development of Globus toolkit • Core services for grid-enabled tools & applns • Construction of large grid testbed: GUSTO • Largest grid testbed in terms of sites & apps • Application experiments • Tele-immersion, distributed computing, etc.

  11. Globus Toolkit Authentication, resource management, information, communication, fault detection, data access, other services as and when available Deployment Subsets of Globus toolkit technology deployed as production infrastructure The Globus Project and the IGA Examples: SF-Express, X-ray source data, astrophysics, crystallography, collaborative design, remote quantum chem. Ongoing: HEP & climate data, combustion corridor, remote Allegra, distributed AMR, CorridorOne, HotPage, ... Applns … rich variety of applications ... Collaborative efforts: e.g., data grid, remote viz, commodity grid, Message Passing Interface, etc., etc., etc. Supported partners: e.g., tele- immersion, parameter studies, debuggers, desktop tools, PSEs, couplers, etc., etc., etc. Appln Toolkits Remote data toolkit Async. collab. toolkit Remote sensors toolkit Remote comp. toolkit Remote viz toolkit ... Research Security, resource mgmt, instrumentation, adaptivity policy, etc., etc. Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Grid Services Grid Fabric Archives, networks, computers, display devices, etc.; associated local services

  12. GUSTO Testbed Map

  13. Globus Grid Services • The Globus toolkit provides a range of basic Grid services • Security, information, fault detection, communication, resource management, ... • These services are simple and orthogonal • Can be used independently, mix and match • Programming model independent • For each there are well-defined APIs • Standards are used extensively • E.g., LDAP, GSS-API, X.509, ...

  14. Grid Security Infrastructure • Single-sign on, run anywhere [if authorized] • PKI, X.509 certificates • Identity/credential mapping at each resource • Allows programs to act as user for limited period: delegation of rights

  15. Technical Details: Grid SecurityServices Provide Single Sign-On User User Proxy Site 1 Process Process GRAM GRAM GSI GSI Process Process Ticket Process Process Public Key Kerberos Single sign-onvia “globus-id” CREDENTIAL Assignment of credentials to “user proxies” Globus Credential Mutual user-resource authentication Site 2 Mapping to local ids Authenticated interprocess communication GSS-API: multiple low-level mechanisms Certificate

  16. “10 GFlops, EOS data, 20 Mb/sec -- for 20 mins” GRAM GRAM GRAM An Example of Globus Services at Work:Resource Management Architecture Info service: location + selection Metacomputing Directory Service Resource Broker “What computers?” “What speed?” “When available?” “20 Mb/sec” GRAM Globus Resource Allocation Managers “50 processors + storage from 10:20 to 10:40 pm” Fork LSF EASYLL Condor etc.

  17. Globus and the National Grid • Selected “Grid Services” are being migrated into the infrastructure • Grid information service • Grid security infrastructure • Grid resource management services • Simultaneously these and other Globus services are being applied to develop • Grid-enabled tools • Grid-enabled applications • An ongoing iterative refinement process

  18. DiffServ Measurement Servers Wisc. Other Potential University Participants ANL ESnet Abilene NWU DiffServ Policy Tools AADS Switch supporting MREN PVCs UIUC UIC DiffServ Enabled Router UChicago The Alliance National Technology Grid EMERGE MREN/ESnet Testbed Globus Project Grid Testbed Partnerships Include ... NASA’s Information Power Grid

  19. Case Study:Online Instrumentation Advanced Photon Source wide-area dissemination desktop & VR clients with shared controls real-time collection archival storage tomographic reconstruction DOE X-ray source grand challenge: ANL, USC/ISI, NIST, U.Chicago

  20. Current Technology Focus Areas • Advanced resource management techniques • GARA: Globus Arch. for Resv. & Allocation • High-end data-intensive applications • “Data Grid” • Interfaces to commodity technologies • CoG-Kit: Commodity Grid Toolkits • Distance visualization • NOVA: Network Optimized Visualization Arch. With supporting work on info/instr., policy, accounting, authentication/authorization, etc.

  21. GARA and Differentiated Services Server Client GARA API Diffserv Resource Manager Diffserv Resource Manager

  22. Scheduling Bulk Transferand High-Priority Transfers

  23. The Grid Forumhttp://www.gridforum.org • IETF-like community forum for discussion & definition of Grid infrastructure • First two meetings (June 16-18, Oct 18-20) attracted 150 people • 9 working groups established in security, information infrastructure, resource management, accounting, etc. • Next mtg: San Diego Feb/Mar 2000 • See also European Grid Forum • www.egrid.org

  24. Summary: Globus Contributions • Architecture that addresses inter-domain issues • Toolkit model for building grid-enabled tools and applications • Information-rich approach to adaptivity and high performance • New technologies for security, resource management, etc. • Construction of production environments and testbeds allowing realistic evaluation

  25. Summary • Grids require Grid services that make resources accessible and usable and Grid toolkits for application development • The Globus project is building essential services and partnering with tool developers • Widespread adoption by groups building production infrastructure • Significant application success stories in a range of problem classes

  26. For More Information ... • Globus: www.globus.org • Grid Forum: www.gridforum.org • Grid book “The Grid: Blueprint for a Future Computing Infrastructure,” I. Foster & C. Kesselman (Eds), Morgan-Kaufmann, 1999 http://www.mkp.com/grids

More Related