National e-Science Core Programme & Grid Highlights BiGUM1 Meeting @ eSI 30 th October 2001

1. National e-Science Core Programme & Grid Highlights BiGUM1 Meeting @ eSI 30th October 2001

2. Contents • Welcome • NeSC and e-Science Support • Grid Definitions • Grid Examples • Grid Architectures

3. e-Science Programme DG Research Councils Grid TAG E-Science Steering Committee Director Director’s Management Role Director’s Awareness and Co-ordination Role Generic Challenges EPSRC (£15m), DTI (£15m) Academic Application Support Programme Research Councils (£74m), DTI (£5m) PPARC (£26m) BBSRC (£8m) MRC (£8m) NERC (£7m) ESRC (£3m) EPSRC (£17m) CLRC (£5m) £80m Collaborative projects Industrial Collaboration (£40m) From Tony Hey 27 July 01

4. UK Grid Network Edinburgh Glasgow DL Newcastle Belfast Manchester Cambridge Oxford Hinxton RAL Cardiff London Soton From Tony Hey 27 July 01

5. Key Elements of UK Grid Development Plan • Network of Grid Core Programme e-Science Centres • Development of Generic Grid Middleware • Grid IRC Grand Challenge Project • Support for e-Science Testbeds • International Involvement via GGF • Grid Network Team From Tony Hey 27 July 01

6. e-Science Centres Application Pilots IRCs … e-Scientists, Grid users, Grid services & Grid Developers NeSC GNT DBTF ATF TAG eSI CS Research GSC UK Core Programme Team Global Grid Forum … NeSC’s context Coordination

7. NeSC — The Team • Director • Malcolm Atkinson (Universities of Glasgow & Edinburgh) • Deputy Director • Arthur Trew (Director EPCC) • Commercial Director • Mark Parsons (EPCC) • Regional Director • Stuart Anderson (Edinburgh Informatics) • Chairman • Richard Kenway (Edinburgh Physics & Astronomy) • Initial Board Members • Muffy Calder (Glasgow Computing Science) • Tony Doyle (Glasgow Physics & Astronomy) • Centre Manager • Anna Kenway

8. NeSC’s Roles • Stimulation of Grid & e-Science Activity • Users, developers, researchers • Education, Training, Support • Think Tank & Research • Coordination of Grid & e-Science Activity • Regional Centres, Task Forces, Pilots & IRCs • Technical and Managerial Fora • Support for training, travel, participation • Developing a High-Profile Institute • Meetings • Visiting Researchers • Regional Support • Portfolio of Industrial Research Projects

9. eSI Highlights • Report given yesterday • History • 3 workshops week 1: DF1, GUM1 & DBAG1 • HEC • preGGF3 & DF2 • October • Steve Tuecke Globus tutorial (oversubscribed) • 4-day workshop Getting Going with Globus (G3) • Reports on DataGrid & GridPP experience • Biologist Grid Users’ Meeting 1 (BiGUM1) • November • GridPP • Configuration management • December • AstroGrid

10. eSI Highlights cont. • 2002 & 2003 • January • Steve Tuecke 4 day Globus Developers’ Workshop • February • UKOLN • March • Protein folding Workshop 14th to 17th IBM sponsor • May • Mind and Brain Workshop • 22nd to 26th July GGF5 & HPDC 11 EICC • August Research Festival • 4 juxtaposed 1-week in-depth workshops • Topics under consideration • Dependability and Security for the Grid • Metadata and the Grid • Provenance, Annotation and Archiving • The Knowledge Grid • Programming Models for the Grid • 14th to 16thApril 2003 Dependability

11. Motivation for IPG • Large-scale science and engineering are done through the interaction of people, heterogeneous computing resources, information systems, and instruments, all of which are geographically and organizationally dispersed. • The overall motivation for “Grids” is to facilitate the routine interactions of these resources in order to support large-scale science and engineering. From Bill Johnston 27 July 01

12. Why Grids? • A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour • 1,000 physicists worldwide pool resources for petaop analyses of petabytes of data • Civil engineers collaborate to design, execute, & analyze shake table experiments • Climate scientists visualize, annotate, & analyze terabyte simulation datasets • An emergency response team couples real time data, weather model, population data From Steve Tuecke 12 Oct. 01

13. Why Grids? (contd.) • A multidisciplinary analysis in aerospace couples code and data in four companies • A home user invokes architectural design functions at an application service provider • An application service provider purchases cycles from compute cycle providers • Scientists working for a multinational soap company design a new product • A community group pools members’ PCs to analyze alternative designs for a local road From Steve Tuecke 12 Oct. 01

14. The Grid Problem • Flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resource From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations” • Enable communities (“virtual organizations”) to share geographically distributed resources as they pursue common goals -- assuming the absence of… • central location, • central control, • omniscience, • existing trust relationships. From Steve Tuecke 12 Oct. 01

15. Elements of the Problem • Resource sharing • Computers, storage, sensors, networks, … • Sharing always conditional: issues of trust, policy, negotiation, payment, … • Coordinated problem solving • Beyond client-server: distributed data analysis, computation, collaboration, … • Dynamic, multi-institutional virtual organisations • Community overlays on classic org structures • Large or small, static or dynamic From Steve Tuecke 12 Oct. 01

16. Why Now? • Moore’s law improvements in computing produce highly functional endsystems • The Internet and burgeoning wired and wireless provide universal connectivity • Changing modes of working and problem solving emphasize teamwork, computation • Network exponentials produce dramatic changes in geometry and geography From Steve Tuecke 12 Oct. 01

17. Network Exponentials • Network vs. computer performance • Computer speed doubles every 18 months • Network speed doubles every 9 months • Difference = order of magnitude per 5 years • 1986 to 2000 • Computers: x 500 • Networks: x 340,000 • 2001 to 2010 • Computers: x 60 • Networks: x 4000 Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan-2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and Perkins. From Steve Tuecke 12 Oct. 01

18. Broader Context • “Grid Computing” has much in common with major industrial thrusts • Business-to-business, Peer-to-peer, Application Service Providers, Storage Service Providers, Distributed Computing, Internet Computing… • Sharing issues not adequately addressed by existing technologies • Complicated requirements: “run program X at site Y subject to community policy P, providing access to data at Z according to policy Q” • High performance: unique demands of advanced & high-performance systems From Steve Tuecke 12 Oct. 01

19. The Globus Project™Making Grid computing a reality • Close collaboration with real Grid projects in science and industry • Development and promotion of standard Grid protocols to enable interoperability and shared infrastructure • Development and promotion of standard Grid software APIs and SDKs to enable portability and code sharing • The Globus Toolkit™: Open source, reference software base for building grid infrastructure and applications • Global Grid Forum: Development of standard protocols and APIs for Grid computing From Steve Tuecke 12 Oct. 01

20. Online Access to Scientific Instruments Advanced Photon Source wide-area dissemination desktop & VR clients with shared controls real-time collection archival storage tomographic reconstruction DOE X-ray grand challenge: ANL, USC/ISI, NIST, U.Chicago From Steve Tuecke 12 Oct. 01

21. Supernova Cosmology Requires Complex,Widely Distributed Workflow Management

22. Mathematicians Solve NUG30 • Looking for the solution to the NUG30 quadratic assignment problem • An informal collaboration of mathematicians and computer scientists • Condor-G delivered 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites) • 14,5,28,24,1,3,16,15, • 10,9,21,2,4,29,25,22, • 13,26,17,30,6,20,19, • 8,18,7,27,12,11,23 MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin From Miron Livny 7 Aug. 01

23. Network for EarthquakeEngineering Simulation • NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other • On-demand access to experiments, data streams, computing, archives, collaboration NEESgrid: Argonne, Michigan, NCSA, UIUC, USC From Steve Tuecke 12 Oct. 01

24. Home ComputersEvaluate AIDS Drugs • Community = • 1000s of home computer users • Philanthropic computing vendor (Entropia) • Research group (Scripps) • Common goal= advance AIDS research From Steve Tuecke 12 Oct. 01

25. Application Application Internet Protocol Architecture “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Collective “Sharing single resources”: negotiating access, controlling use Resource “Talking to things”: communication (Internet protocols) & security Connectivity Transport Internet “Controlling things locally”: Access to, & control of, resources Fabric Link Layered Grid Architecture(By Analogy to Internet Architecture) From Steve Tuecke 12 Oct. 01

26. Architecture of a Grid Discipline Specific Portals andScientific Workflow Management Systems Applications: Simulations, Data Analysis, etc. Toolkits: Visualization, Data Publication/Subscription, etc. Grid Common Services: Standardized Services and Resources Interfaces Collaboration and Remote Instrument Services Grid Information Service UniformResourceAccess Co-Scheduling Network Cache Authentication Authorization Security Services Communication Services Global Queuing Global EventServices Data Cataloguing Uniform Data Access Fault Management Monitoring Brokering Auditing = Globus services clusters Distributed Resources national supercomputer facilities tertiary storage national user facilities Condor pools networkcaches High-speed Networks and Communications Services

27. data publish and subscribe toolkits instrument management toolkits visualization toolkits collaboration toolkits application codes Condor-G Java/Jini Globus MPI CORBA DCOM Architecture of a Grid – upper layers • Knowledge based query • Tools to implement the human interfaces, e.g. SciRun, ECCE, WebFlow, ..... • Mechanisms to express, organize, and manage the workflow of problem solutions (“frameworks”) • Access control Problem Solving Environments Applications and Supporting Tools Grid enabled libraries (security, communication services, data access, global event management, etc.) Application Development and Execution Support Grid Common Services From Steve Tuecke 12 Oct. 01 Distributed Resources

28. Knowledge Grid Control Data to Knowledge Information Grid Computation/ Data Grid Three Layer GRID Abstraction From Tony Hey 12 Sep. 01

29. Data, Information and Knowledge • Data Uninterpreted bits and bytes • Information Data equipped with meaning • Knowledge Information applied to achieve a goal, solve a problem or enact a decision From Tony Hey 12 Sep. 01

30. Biological Grid Users • Are they different? • Do they have different collaborations? • Do they have different data? • Do they have different computations? • Do they have the same shared “instruments”? • Can they be supported using the same • Infrastructure • Architecture • Policies?