MSA’2000 Metacomputing Systems and Applications

1. MSA’2000Metacomputing Systems and Applications

2. Organizing Committee • F. Desprez, INRIA Rhône-Alpes • E. Fleury, INRIA Lorraine • J.-F. Méhaut, INRIA Rhône-Alpes • Y. Robert, ENS Lyon • www.ens-lyon.fr/LIP/ MSA Introduction

3. Program Committee OC + • H. Bal, Vrije University, Amsterdam • F. Berman, UCSD San Diego • J. Dongarra, UT Knoxville & ONRL • G. von Laszewski, Argonne • T. Ludwig, TUM München • T. Priol, INRIA Rennes • M. Resch, Stuttgart MSA Introduction

4. The Grid: Blueprint for a New Computing Infrastructure I. Foster, C. Kesselman (Eds), Morgan Kaufmann, 1999 • ISBN 1-55860-475-8 • 22 chapters by expert authors including Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others MSA Introduction

5. Bibliography • Web • NPCACI (National Partnership for Advanced Computational Infrastructure) www.npaci.edu • GrADS (Grid Application Development Software Project) hipersoft.cs.rice.edu/grads • “An Overview of Computational Grids and Survey of a Few Research Projects”, Jack Dongarrawww.netlib.org/utk/people/JackDongarra/talks.html • LIP Report 99-36 • “Algorithms and Tools for (Distributed) Heterogeneous Computing: A Prospective Report” www.ens-lyon.fr/~yrobert MSA Introduction

6. Framework

7. Metacomputing • Future of parallel computingdistributed and heterogeneous • Metacomputing=Making use of distributed collections of heterogeneous platforms • Target= Tightly-coupled high-performance distributed applications(rather than loosely-coupled cooperative applications) MSA Introduction

8. Metacomputing Platforms (1) • Low end of the fieldCluster computing with heterogeneous networks of workstations or PCs • Ubiquitous in university departments and companies • Typical poor man’s parallel computer • Running large PVM or MPI experiments • Make use of all available resources: slower machinesin addition to more recent ones MSA Introduction

9. Metacomputing Platforms (2) • High end of the fieldComputational grid linking the most powerful supercomputers of the largest supercomputing centers through dedicated high-speed networks. • Middle of the fieldConnecting medium size parallel servers (equipped with application-specific databases and application-oriented software) through fast but non-dedicated, thus creating a “meta-system” MSA Introduction

10. High end: Gusto MSA Introduction

11. Low end (1) • Distributed ASCI Supercomputer (DAS) • Common platform for research • (Wide-area) parallel computing and distributed applications • November 1998, 4 universities, 200 nodes • Node • 200 MHz Pentium Pro • 128 MB memory, 2.5 GB disk • Myrinet 1.28 Gbit/s (full duplex) • Operating System: BSD/OS • ATM Network MSA Introduction

12. Low end (2) MSA Introduction

13. Administrative Issues • Intensive computations on a set of processors across several countries and institutions • Strict rules to define the (good) usage of shared resources • A major difficulty is to avoid a large increase in the administrative overhead • Challenge = find a tradeoff that does not increase the administrative load while preserving the users’ security se rules must be guaranteed by the runtime, together with methods to migrate computations to other sites whenever some local request is raised MSA Introduction

14. Tomorrow’s Virtual Super-Computer • Metacomputing applications will execute on a hierarchical grid • Interconnection of clusters scattered all around the world • A fundamental characteristic of the virtual super-computer: • A set of strongly heterogeneous and geographically scattered resources MSA Introduction

15. Algorithmic and Software Issues (1) Whereas the architectural vision is clear,the software developments are not so well understood MSA Introduction

16. Algorithmic and Software Issues (2) • Low end of the field: • Cope with heterogeneity • Major algorithmic effort to be undertaken • High end of the field • Logically assemble the distributed computers: extensions to PVM and MPI to handle distributed collection of clusters • Configuration and performance optimization • Inherent complexity of networked and heterogeneous systems • Resources often identified at runtime • Dynamic nature of resource characteristics MSA Introduction

17. Algorithmic and Software Issues (3) • High-performance computing applications must: • Configure themselves to fit the execution environment • Adapt their behavior to subsequent changes in resource characteristics • Parallel environments focused on strongly homogeneous architectures (processor, memory, network) • Array and loop distribution, parallelizing compilers, HPF constructs, gang scheduling, MPI However… Metacomputing platforms are strongly heterogeneous! MSA Introduction

18. Programming environments

19. Programing models (1) • Extensions of MPI: • MPI_Connect, Nexus, PACX-MPI, MPI-Plus, Data-Exchange, VCM, MagPIe, … • Globus: a layered approach • Fundamental layer = a set of core services, including resource management, security, and communications that enable the linking and interoperation of distributed computer systems MSA Introduction

20. Programing models (2) • Object-oriented technologies to cope with heterogeneity: • Encapsulate technical ``details'' such as protocols, data representations, migration policies • Legion is building on Mentat, an object-oriented parallel processing system • Albatross relies on a high-performance Java system, with a very efficient implementation of Java Remote Method Invocation. MSA Introduction

21. Programing models (3) • Far from achieving the holy goal: • Using the computing resources remotely and transparently,just as we do with electricity,without knowing where it comes from MSA Introduction

22. References • Globus www.globus.org • Legion www.cs.virginia.org/~legion • Albatross www.cs.vu.nl/~bal/albatross • AppLeSwww-cse.ucsd.edu/groups/hpcl/apples/apples.html • NetSolve www.cs.utk.edu/netsolve MSA Introduction

23. Algorithmic issues

24. Data Decomposition Techniques for Cluster Computing • Block-cyclic distribution paradigm = preferred layout for data-parallel programs (HPF, ScaLAPACK) • Evenly balances total workload only if all processors have same speed • Extending ScaLAPACK to heterogeneous clusters turns out to be surprisingly difficult MSA Introduction

25. Algorithmic challenge • Bad news: designing a matrix-matrix product or a dense linear solver proves a hard task on a heterogeneous cluster! • Next problems: • Simple linear algebra kernels on a collection of clusters (extending the platform) • More ambitious routines, composed of a variety of elementary kernels, on a heterogeneous cluster (extending the application) • Implementing more ambitious routines on more ambitious platforms (extending both) MSA Introduction

26. Collections of clusters (1) Fast link Slower link MSA Introduction

27. Conclusion

28. (A) Algorithmic issues • Difficulties seem largely underestimated • Data decomposition, scheduling heuristics, load balancing become extremely difficult in the context of metacomputing platforms • Research community focuses on low-level communication protocols and distributed system issues (light-weight process invocation, migration, ...) MSA Introduction

29. (B) Programming level • Which is the good level ? • Data-parallelism unrealistic, due to heterogeneity • Explicit message passing too low-level • Object-oriented approaches still request the user to have a deep knowledge of both its application behavior and the underlying resources • Remote computing systems (NetSolve) face severe limitations to efficiently load-balance the work • Relying on specialized but highly-tuned libraries of all kinds may prove a good trade-off MSA Introduction

30. (C) Applications • Key applications (from scientific computing to data-bases) have dictated the way classical parallel machines are used, programmed, and even updated into more efficient platforms • Key applications will strongly influence, or even guide, the development of metacomputing environments MSA Introduction

31. (C) Applications (cont’d) • Which applications will be worth the abundant but hard-to-access resources of the grid ? • tightly-coupled grand challenges ? • mobile computing applications ? • micro-transactions on the Web ? • All these applications require new programming paradigms to enable inexperienced users to access the magic grid! MSA Introduction

32. Today’s program

33. Session 1: Communication and Metacomputing Infrastructures • 9h00:10h00, Metacomputing in a High Performance Computing Center (invited talk), M. Resh. • 10:30-11:00, Scheduling Algorithms for Efficient Gather Operation in Distributed Heterogeneous Systems,Juin-ichi Hatta & Susumu Shibusawa • 11:00-11:30, Applying and Monitoring Latency Based Metacomputing Infrastructures, Philipp Drum & Günther Rackl. • 11:30-12:00, MPC: A New Message Passing Library in CorbaT. Es-sqally, J. Guyard & E. Fleury. MSA Introduction

34. Session 2: Scientific Applications and Distributed Computing • 14:00-15:00, The Netsolve Environment: Processing Towards a Seamless Grid (invited talk), D. Arnold & J. Dongarra • 15:30-16:00, Specification of a Scilab Meta-Computing Extension,S. Contassot-Vivier, F. Lombard, J-M. Nicod & L. Philippe • 16:00-16:30, Extending WebCom: A Proposed Framework for Web based Distributed Computing, J. P. Morrison, J. J. Kennedy & D. A. Power • 16:30-17:30, Panel discussion MSA Introduction