Grid Computing

1. Grid Computing The 21st Century Paradigm for Service-Oriented Utility Computing 2004 http://www.info.uvt.ro/~petcu/grid.ppt

2. Overview • What is Grid? • Grid Projects & Applications • Grid Technologies • Globus • CompGrid

3. What is Grid?

6. Definition A type of parallel and distributed system that enables the sharing, selection, & aggregation of geographically distributed resources: • Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc; • Software – e.g., ASPs renting expensive special purpose applications on demand; • Catalogued data and databases – e.g. transparent access to human genome database; • Special devices/instruments – e.g., radio telescope – SETI@Home searching for life in galaxy. • People/collaborators. depending on their availability, capability, cost, and user QoS requirements for solving large-scale problems/applications. thus enabling the creation of “virtual organization” (VOs)

7. Distributed supercomputing support • High-throughput computing support • On-demand computing support • Data-intensive computing support • Collaborative computing support • Multimedia computing support • ....

9. Resources = assets, capabilities, and knowledge • Capabilities (e.g. application codes, analysis tools) • Compute Grids (PC cycles, commodityclusters, HPC) • Data Grids • Experimental Instruments • Knowledge Services • Virtual Organisations • Utility Services

10. Grid‘s main idea • To treat CPU cycles and software like commodities. • Enable the coordinated use of geographically distributedresources – in the absence of central control and existingtrust relationships. • Computing power is produced much like utilities such aspower and water are produced for consumers. • Users will have access to “power” on demand • “When the Network is as fast as the computer’s internallinks, the machine disintegrates across the Net into a set ofspecial purpose appliances”– Gilder Technology Report June 2000

11. Computational Grids andElectric Power Grids • Power Grid analogy – Power producers: machines, software, networks,storage systems – Power consumers: user applications • Applications draw power from the Grid the wayappliances draw electricity from the power utility. – Seamless, High-performance, Ubiquitous, Dependable • Why theComputational Grid islike the Electric PowerGrid – Electric power isubiquitous – Don’t need to knowthe source of thepower (transformer,generator) or thepower company thatserves it • Why theComputational Grid isdifferent from theElectric Power Grid – Wider spectrum ofperformance – Wider spectrum ofservices – Access governed bymore complicatedissues: Security, Performance

15. Distributed Computing • Concept has been around for two decades • Basic idea: run scheduler across systems to runs processes on least- used systemsfirst – Maximize utilization – Minimize turnaround time • Have to load executables and input files toselected resource – Shared file system – File transfers upon resource selection

16. Examples of DistributedComputing • Workstation farms etc – Generally share file system • SETI@home project, Entropia, etc. – Only one source code; copies correct binarycode and input data to each system • Napster, Gnutella: file/data sharing • NetSolve – Runs numerical kernel on any of multipleindependent systems, much like a Gridsolution

22. Internet Computing Projects

26. Why cluster and grid computing • Clusters and grids increasingly interesting • more workstations • higher performance per workstation • faster interconnecting networks • price/performance competitive with MPP • enormous unused capacity • cyclic availability

27. Differences parallel/clusters/grids • Clusters are inherently inhomogeneous • intrinsic differences in performance, memory, bandwidth • dynamically changing “background load” • ownership of nodes • Grids add • differences in administration • disjoint file systems • security etc.

28. P2P, cluster, Internet computing vs. grid computing • Peer-to-peer networks (eg Kazaa) fall within the definition of grid computing (the resource shared is the storage capacity of each node) P2P Working Group part of Global Grid Forum • A cluster is a resource that can be shared- a grid is a cluster of clusters • Internet computing: a VO is assembled for a particular project and disbanded once the project is complete -the shared resource is the Internet connected desktop

37. Why go Grid? • Hot subject • Try it, experience it to learn the potential • Will enable true ubiquitous computing in future • Today, proven in some areas: intraGrids • But still long way to World Wide Grid • State of art techniques, tools are difficult • Short term goals? Use another technology • Does your system have Grid characteristics? • Distributed users, large scale and heterogeneous resources, across domains

38. Why? • Grids enable much more than appsrunning on multiple computers – virtual operating system: provides globalworkspace/address space via a single login – automatically manages files, data, accounts,and security issues – connects other resources (archival datafacilities, instruments, devices) and people(collaborative environments) • Inevitable (at least in HPC): – leverages computational power of all availablesystems – manages resources as a single system--easier forusers – provides most flexible resource selection andmanagement, load sharing – researchers’ desire to solve bigger problems willalways outpace performance increases of singlesystems; just as multiple processors are needed,‘multiple multiprocessors’ will be deemed so

39. Why? • Resources have different functions, butmultiple classes resources are necessaryfor most interesting problems. • Power of any single resource is smallcompared to aggregations of resources • Network connectivity is increasing rapidlyin bandwidth and availability • Large problems require teamwork andcomputation

40. What do users want ? • Grid Consumers • Execute jobs for solving varying problem size and complexity • Benefit by selecting and aggregating resources wisely • Tradeoff timeframe and cost • Grid Providers • Contribute (“idle”) resource for executing consumer jobs • Benefit by maximizing resource utilisation • Tradeoff local requirements & market opportunity

44. Grid projects & applications

45. Proiecte vechi • 1996: Ian Foster, Steven Tuecke si Carl Kesselman de la ANL–SUA, infrastructurii de interconectare a celor mai importante centre de calcul de inalta performanta, proiectul I-WAY. • 1998: comunitatii de utilizatori internationali si de standarde in grid, Global Grid Forum. • RO: 2002 si 2003 cateva proiecte in domeniul grid prin programul InfoSoc • EU-FP5: 2000/2001: EuroGrid, DataGrid si Damien (infrastructura,middleware: Geant; Unicore). • EU-FP5: 2001/2002: middleware, aplicatii; GridLab – platforma de testare, CrossGrid – interoperabilitate griduri, simulari, EGSO – astro-fizica, GRIA – industrial, DataTag – platforma transatlantica, GRIP - interoperabilitate. • EU-FP5: 2002/2003: aplicatii; Avo – astro-fizica, FlowGrid - simulare, OpenMolGrid – molecular, GRACE – cautare, COG – ontologii, MOSES – web semantic, BioGrid – bilogic, GEMSS – medical, SeLeNe – e-learning, MamoGrid – medical, EGEE – securitate, NOMAD – descoperire de servicii (aprox 20 proiecte) • EU-FP6: 2003/2004, ‘Grid for complex problem solving’ • proiecte nationale: UK - e-Science (80 proiecte) incluzand GridPP, Comb-e-Grid, AstroGrid, MyGrid, GEODISC, DAME, DiscoveryNet, RealityGrid, OGSA-DA; Franta, la INRIA, ruleaza o serie de proiecte: Algorille – management de resurse pe grid, Apache – planificare multicriteriala, Grand – desktop grid, Oasis – grid de increder, Paris – simulari numerice, Remap – servere in retea, Sardies – monitorizare, MPICH-V – MPI pentru grid. In alte tari: Japonia – Grid Data Farm, ITBL, Olanda – VLAM, DutchGrid, Italia – INFN Grid, Irelanda – EireGrid, Polonia – PIONIERGrid, Ungaria – DemoGrid, JiniGrid, Australian –SimGrid, Economy Grid., WWG • USA: NASA Information Power Grid, DOE Science Grid, NSF National Virtual Observatory, NSF GriPhyN, DOE Particle Physics Data Grid, NSF DTF TeraGrid, DOE ASCI DISCOM Grid, DOE Earth Systems Grid, DOE FusionGrid, NEESGrid, NIH BIRN, NSF iVDGL. • IBM a realizat un Grid Toolkit bazat pe Globus, iar Sun, un One-Grid-Engine.

46. Maturation of Grid Computing • Research focus moving from building of basicinfrastructure and application demonstrations to – Middleware – Usable production environments – Application performance – Scalability -> Globalization • Development, research, and integration happeningoutside of the original infrastructure groups • Grids becoming a first-class tool for scientificcommunities – GriPhyN (Physics), BIRN (Neuroscience), NVO(Astronomy), • Widespread interest from government indeveloping computational Grid platforms; in US NSF’s Cyberinfrastructure NASA’s Information Power Grid DOE’s Science Grid

47. Grid Applications • Distributed HPC (Supercomputing): • Computational science. • High-Capacity/Throughput Computing: • Large scale simulation/chip design & parameter studies. • Content Sharing (free or paid) • Sharing digital contents among peers (e.g., Napster) • Remote software access/renting services: • Application service provides (ASPs) & Web services. • Data-intensive computing: • Drug Design, Particle Physics, Stock Prediction... • On-demand, real-time computing: • Medical instrumentation & Mission Critical. • Collaborative Computing: • Collaborative design, Data exploration, education. • Service Oriented Computing (SOC): • Towards economic-based Utility Computing: New paradigm, new applications, new industries, and new business.

48. Australia Nimrod-G Gridbus GridSim Virtual Lab DISCWorld GrangeNet ..new coming up Europe UNICORE Cactus UK eScience EU Data Grid EuroGrid MetaMPI XtremeWeb and many more. India I-Grid Japan Ninf DataFarm Korea... N*Grid USA Globus Legion OGSA Sun Grid Engine AppLeS NASA IPG Condor-G Jxta NetSolve AccessGrid and many more... Cycle Stealing & .com Initiatives Distributed.net SETI@Home, …. Entropia, UD, Parabon,…. Public Forums Global Grid Forum Australian Grid Forum IEEE TFCC CCGrid conference P2P conference Grid Projects

49. NASA’s IPG Vision for the Information Power Grid is topromote a revolution in how NASA addresseslarge-scale science and engineering problemsby providing persistent infrastructure for – “highly capable” computing and data managementservices that, on-demand, will locate and coschedulethe multi-Center resources needed toaddress large-scale and/or widely distributedproblems – the ancillary services that are needed to supportthe workflow management frameworks thatcoordinate the processes of distributed science andengineering problems