Peer to Peer & Grid Computing

1. Peer to Peer & Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster Talk at Internet2 Peer to Peer Workshop, January 30, 2002

2. The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

3. Grids and Peer-to-Peer Computing • Common interests • Enabling the coordinated use of geographically distributed resources—in the absence of central control and existing trust relationships • Authentication, authorization, auditing, resource discovery, resource management, remote data access, fault tolerance, … • “Beyond client server” • Differences • Historically high-end vs. strongly commodity • Predominately eScience vs. eCommodity

4. The Grid World: Current Status • Dozens of major Grid projects in scientific & technical computing/research & education • Deployment, application, technology • Consensus on key concepts & technologies • Open source Globus Toolkit™ a de facto standard for major protocols & services • Far from complete or perfect, but out there, evolving rapidly, and large tool/user base • Global Grid Forum a significant force • Industrial interest emerging rapidly • Commercial support for Globus Toolkit; OGSA

5. Why Grids? (1) eScience • A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour • 1,000 physicists worldwide pool resources for peta-op analyses of petabytes of data • Civil engineers collaborate to design, execute, & analyze shake table experiments • Climate scientists visualize, annotate, & analyze terabyte simulation datasets • An emergency response team couples real time data, weather model, population data

6. ~PBytes/sec ~100 MBytes/sec Offline Processor Farm ~20 TIPS There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size ~100 MBytes/sec Online System Tier 0 CERN Computer Centre ~622 Mbits/sec or Air Freight (deprecated) Tier 1 FermiLab ~4 TIPS France Regional Centre Germany Regional Centre Italy Regional Centre ~622 Mbits/sec Tier 2 Tier2 Centre ~1 TIPS Caltech ~1 TIPS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS Tier2 Centre ~1 TIPS HPSS HPSS HPSS HPSS HPSS ~622 Mbits/sec Institute ~0.25TIPS Institute Institute Institute Physics data cache ~1 MBytes/sec 1 TIPS is approximately 25,000 SpecInt95 equivalents Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz Pentium II 300 MHz Tier 4 Physicist workstations Grid Communities & Applications:Data Grids for High Energy Physics www.griphyn.org www.ppdg.net www.eu-datagrid.org

7. Grid Physics Network (GriPhyN) Enabling R&D for advanced data grid systems, focusing in particular on Virtual Data concept ATLAS CMS LIGO SDSS Paul Avery, Ian Foster, Co-PIs www.griphyn.org

8. Grid Communities and Applications:Network for Earthquake Eng. Simulation • NEESgrid: US national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other • On-demand access to experiments, data streams, computing, archives, collaboration NEESgrid: Argonne, Michigan, NCSA, UIUC, USC www.neesgrid.org

9. Presenter mic Presenter camera Ambient mic (tabletop) Audience camera Example Grid Communities:Access Grid Collaboration • Enable collaborative work at dozens of sites worldwide, with strong sense of shared presence • Combination of commodity audio/video tech + Grid technologies for security, discovery, etc. • 40+ sites worldwide, number rising rapidly

10. Why Grids? (2) eBusiness • Engineers at a multinational company collaborate on the design of a new product • A multidisciplinary analysis in aerospace couples code and data in four companies • An insurance company mines data from partner hospitals for fraud detection • An application service provider offloads excess load to a compute cycle provider • An enterprise configures internal & external resources to support eBusiness workload

11. Open Grid Services Architecture(ANL-IBM-ISI; Globus Toolkit 3.0) • Service orientation to virtualize resources • From Web services (WSDL, WS-Ins, etc.): • Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency • Building on Globus Toolkit: • Grid service: semantics for service interactions • Management of transient instances (& state) • Factory, Registry, Discovery, other services • Reliable and secure transport • Multiple hosting targets: J2EE, .NET, “C”, …

12. Grids and Universities • “What can universities do to enhance their constituents’ abilities to form & participate in virtual organizations?” • Deploy standard central services: authentication, directories; policies • Create campus Grids offering distributed compute services, data services • Provide training and consulting in Grid technologies and applications • NSF middleware initiative offers support for such activities

13. NSF Middleware Initiative (NMI)nsf-middleware.org • GRIDS (Grid Research Integration Development & Support) Center www.grids-center.org • Integration, packaging, testing, distribution of standard NMI software (security, directory, compute, storage, etc., etc.) • Training, documentation, some support • Single POC for high-quality Grid software • EDIT (I2 + SURA) • Focus on policies and end-user tools

14. For More Information • The Globus Project™ • www.globus.org • GriPhyN • www.griphyn.net • GRIDS Center • www.grids-center.org • Global Grid Forum • www.gridforum.org • Grid architecture • www.globus.org/research/papers/anatomy.pdf