1 / 17

The Grid as Infrastructure and Application Enabler

The Grid as Infrastructure and Application Enabler. Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster. The Grid.

holden
Download Presentation

The Grid as Infrastructure and Application Enabler

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Grid as Infrastructureand Application Enabler Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster

  2. The Grid “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”

  3. Resources • Computing, storage, data A • Connectivity • Reduce tyranny of distance A A • Technologies • Build services & applications Grid Infrastructure • Services • Authentication, discovery, … • Communities • Operational procedures, …

  4. The Grid World: Current Status • Dozens of major Grid projects in scientific & technical computing/research & education • Compute-intensive, data-intensive, remote instrumentation, collaboration, … • Open source Globus Toolkit™ a de facto standard for major protocols & services • Supporting many tools & applns in data-intensive and collaboration-intensive science • Major investments in physical infrastructure • Global Grid Forum: community & standards

  5. Examples ofEmerging Grid Infrastructure • iVDGL: Data-intensive infrastructure • Building a (international) community • Data Grid middleware • Chimera virtual data system • Open Grid Services Architecture • Future service & technology infrastructure

  6. iVDGL: A Global Grid Laboratory “We propose to create, operate and evaluate, over asustained period of time, an international researchlaboratory for data-intensive science.” From NSF proposal, 2001 • International Virtual-Data Grid Laboratory • A global Grid laboratory (US, Europe, Asia, South America, …) • A place to conduct Data Grid tests “at scale” • A mechanism to create common Grid infrastructure • A laboratory for other disciplines to perform Data Grid tests • A focus of outreach efforts to small institutions • U.S. part funded by NSF (2001-2006) • $13.7M (NSF) + $2M (matching)

  7. Tier1 (FNAL) Proto-Tier2 Tier3 university Initial US-iVDGL Data Grid SKC BU Wisconsin PSU BNL Fermilab Hampton Indiana JHU Caltech UCSD Florida Brownsville Other sites to be added in 2002

  8. Tier0/1 facility Tier2 facility Tier3 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link iVDGL:International Virtual Data Grid Laboratory U.S. PIs: Avery, Foster, Gardner, Newman, Szalay www.ivdgl.org

  9. Grid Evolution:Open Grid Services Architecture • Refactor Globus protocol suite to enable common base and expose key capabilities • Secure, reliable invocation; service info; notification; soft state lifetime mgmt; … • Service orientation to virtualize resources and unify resources/services/information • Standard IDL for encapsulation • Embrace key Web services technologies: WSDL as IDL, leverage commercial efforts • And WS Security, WS Routing, etc.

  10. OGSA Structure • A standard substrate: the Grid service • Standard interfaces and behaviors that address key distributed system issues • The “Grid Service Specification” • … supports standard service specifications • Resource management, databases, workflow, security, diagnostics, etc., etc. • Target of current & planned GGF efforts • … and arbitrary application-specific services based on these & other definitions

  11. OGSA Status • Grid service spec near completion in GGF • Globus Toolkit implementation available • IBM & Fujitsu implementations underway • Other companies committed to support it • Various higher-level services underway • OGSI-based Globus Toolkit v3 (GT3), will support GT2 interfaces by end of 2002 • Database services (UK eScience program) • Resource information & management (CIM) • Etc., etc.

  12. Programs as Community Resources:Data Derivation and Provenance • Most [scientific] data are not simple “measurements”; essentially all are: • Computationally corrected/reconstructed • And/or produced by numerical simulation • And thus, as data and computers become ever larger and more expensive: • Programs are significant community resources • So are the executions of those programs • A virtual data system provides a unified view of data, programs, and executions

  13. “I’ve come across some interesting data, but I need to understand the nature of the corrections applied when it was constructed before I can trust it for my purposes.” Virtual Data “I’ve detected a calibration error in an instrument and want to know which derived data to recompute.” Data consumed-by/ generated-by created-by Transformation Derivation execution-of “I want to apply an astronomical analysis program to millions of objects. If the results already exist, I’ll save weeks of computation.” “I want to search an astronomical database for galaxies with certain characteristics. If a program that performs this analysis exists, I won’t have to write one from scratch.”

  14. GriPhyN VDT: Replica catalog DAGMan Globus Toolkit Etc. Chimera Virtual Data System(www.griphyn.org/chimera) • Virtual data catalog • Transformations, derivations, data • Virtual data language • Data definition + query • Applications include browsers and data analysis applications Joint work with Jens Vöckler, Mike Wilde, Yong Zhao

  15. Galaxy cluster size distribution Chimera Virtual Data System + GriPhyN Virtual Data Toolkit + iVDGL Data Grid (many CPUs) Chimera Application:Sloan Digital Sky Survey Analysis Size distribution of galaxy clusters? Joint work with Jim Annis, Steve Kent, FNAL

  16. Summary • “Resource sharing, coordinated problem solving in dynamic, multi-institutional virtual orgs” • Adoption in eScience, transitioning to industry • Emerging physical infrastructure • TeraGrid, iVDGL, DOE Science Grid, …, … • Open Grid Services Architecture • Integrated treatment of major Grid issues • Uniform treatment of resources, data, services • Chimera virtual data system • New abstractions for application development

  17. For More Information • The Globus Project™ • www.globus.org • Technical articles • www.mcs.anl.gov/~foster • TeraGrid, iVDGL • www.teragrid.org, www.ivdgl.org • Open Grid Services Arch. • www.globus.org/ogsa • Chimera • www.griphyn.org/chimera • Global Grid Forum • www.gridforum.org

More Related