Convergence Characteristics for Clusters, Grids, and P2P networks

WW Grid Convergence Characteristics for Clusters, Grids, and P2P networks Panel at the P2P conference, Linkoping, Sweden Grid Computing and Distributed Systems (GRIDS) Lab. The University of MelbourneMelbourne, Australiawww.gridbus.org Rajkumar Buyya

Where Did I come From ? Stockholm+Linkoping * London * Singapore * Melbourne *

GRIDS Lab @ the U. of Melbourne, The Gridbus Project: www.gridbus.org • Conduct R&D in Distributed Computational Economy for Service-Oriented Computing: • Architecture, Specification, and Open Source Reference Implementation in in Collaboration with Any Global Peers. • Platform: Cluster, Grid, P2P for diverse applications. • Distributed Application Composition Tools • Grid Economy & Scheduling (via Nimrod-G Broker) • GridSim Toolkit: Grid Modeling and Simulation (Java based): • Libra: Economic Cluster Scheduler • Grid Bank: Accounting and Transaction Management • World Wide Grid (WWG) – A Global Testbed • P2P/Jxta-based Compute Power Market • Application Enabler Projects: • Virtual Laboratory Toolset for Drug Design • High-Energy Physics and the Grid Network (HEPGrid) • Brain Activity Analysis on the Grid • Future: GridEmail for Spam Management—read only if sender pays!

Books

Technology Evolution: Cluster, Grid, P2P

2100 2100 2100 2100 2100 2100 2100 2100 2100 Scalable Seamless Computing: Breaking Administrative Barriers ? PERFORMANCE Administrative Barriers • Individual • Group • Department • Campus • State • National • Globe • Inter Planet • Galaxy Desktop SMPs or SuperComputers Global Cluster/Grid Inter Planetary Grid! Local Cluster Enterprise Cluster/Grid

Security Data locality Resource Allocation & Scheduling Computational Economy Uniform Access System Management Resource Discovery Network Management Basic Elements Application Development Tools

PC/Workstation PC/Workstation PC/Workstation PC/Workstation Communications Software Communications Software Communications Software Communications Software Network Interface Hardware Network Interface Hardware Network Interface Hardware Network Interface Hardware Cluster Architecture Parallel Applications Parallel Applications Parallel Applications Sequential Applications Sequential Applications Sequential Applications Parallel Programming Environment Cluster Middleware (Single System Image and Availability Infrastructure) Cluster Interconnection Network/Switch

database A Typical Grid Computing Environment Grid Information Service Grid Resource Broker Application R2 2 R3 R4 R5 RN Grid Resource Broker R6 R1 Resource Broker Grid Information Service

A Typical P2P Computing Environment Peer Discovery Service Peer Agent Application P3 pM Who can help ? Peer P2, P7 can help! pN Request P2 Sorry, I am busy. Peer Agent Request Peer Agent Response P1 R7 p4 p5

Cluster, Grid, P2P: Characteristics

Nimrod-G: Modeling Distributed Computing in Economic Terms Nimrod/G Client Nimrod/G Client Nimrod/G Client Nimrod-G Engine Schedule Advisor Trading Manager Grid Store Grid Dispatcher Grid Explorer Grid Middleware Globus, Legion, Condor, etc. TM TS GE GIS Grid Information Server(s) RM & TS RM & TS RM & TS G C L G Legion enabled node. Globus enabled node. L G C L RM: Local Resource Manager, TS: Trade Server Condor enabled node.

On-Demand Computing Cost Deadline Legion hosts Globus Hosts Bezek is in both Globus and Legion Domains

Active Sheet  Nimrod-G:Transparent Spreadsheet Processing from PCs to Clusters to World-Wide Grids Nimrod Proxy Nimrod-G World-Wide Grid

Data Generation Results • [deadline, budget, optimization preference] MEG(MagnetoEncephaloGraphy) Data Analysis on the Grid: Brain Activity Analysis Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs 64 sensors MEG 2 3 1 Data Analysis 5 Nimrod-G 4 Life-electronics laboratory, AIST World-Wide Grid • Provision of expertise in • the analysis of brain function • Provision of MEG analysis [Collaboration with Osaka University, Japan]

Backup Slides

* HTC * P2P * PDAs Minicomputers * * PCs * Workstations * Mainframes * Grids COMPUTING * PC Clusters * Crays * MPPs * WS Clusters * XEROX PARC worm * IETF * W3C * TCP/IP * Ethernet * HTML * Mosaic * Web Services NETWORKING * Email * Sputnik * Internet Era * WWW Era * XML * ARPANET 1960 1970 1975 1980 1985 1990 1995 2000 Evolution Time Line

What is Cluster/Grid ? • A type of parallel and distributed system that enables the sharing, selection, & aggregationof resources distributed in _____/_______ administrative domains depending on their availability, capability, performance, cost, and users quality of service requirements. Multiple A Single Cluster Grid

Market Server Market Repository CPM Agent Bill CPM: DC Economy-based P2P Computing(Jxta based Implementation) • Discovery • - Membership User (Consumer) Trader Job Management Resources (Provider) Accounting

Molecules Protein Drug Design: Data Intensive Computing on Grid • It involves screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential to serve as drug candidates. Chemical Databases (legacy, in .MOL2 format) [Collaboration with WEHI for Medical Science, Melbourne]

GTS GTS GTS GTS Virtual Drug DesignA Virtual Lab for “Molecular Modeling for Drug Design” on P2P Grid Grid Info. Service Grid Market Directory Data Replica Catalogue “Give me list PDBs sources Of type aldrich_300?” “service cost?” “service providers?” GTS Resource Broker “Screen 2K molecules in 30min. for $10” “mol.5 please?” (RB maps suitable Grid nodes and Protein DataBank) “get mol.10 from pdb1 & screen it.” PDB2 “mol.10 please?” (GTS - Grid Trade Server) PDB1

Convergence Characteristics for Clusters, Grids, and P2P networks