260 likes | 388 Views
Components for high performance grid programming in the GRID.IT project. M. Aldinucci M. Coppola D. Laforenza D. Puppin ISTI CNR Pisa. S. Campa M. Danelutto L. Scarponi M. Vanneschi C. Zoccolo Dept. Computer Science University of Pisa. The goal.
E N D
Components for high performance grid programming in the GRID.IT project M. Aldinucci M. Coppola D. Laforenza D. Puppin ISTI CNR Pisa S. Campa M. Danelutto L. Scarponi M. Vanneschi C. Zoccolo Dept. Computer Science University of Pisa Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
The goal • Programming environment supporting the development of • Efficient • Scalable • Fault tolerant • Adaptive • grid aware applications Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Starting from: ASSIST • Structured parallel programming environment • Sequential (C, C++, F77) or parallel modules • Data flow streams (nondeterministic control) • Generic graph compositions • Parallel module:parmod • User defined: input/output section, virtual processors • Specialized to implement most classical and non classical parallelism exploitation patterns Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
The ASSIST track … • ASSIST 1.0 • Targets homogeneous WS clusters (POSIX TCP/IP) • ASSIST 1.2 • Heterogeneous clusters (Linux Pentium / MacOSX PPC) and GRID (Globus 2.4) • ASSIST 2.0 (under development) • GRID, adaptivity, components • … and most of this talk! Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Applications Programming environment Abstract machine GRID middleware The approach: layered architecture • Applications • Complex, multidisciplinary, parallel • High level programming tools • High performance programming environment • Structured, compositional, adaptive • Performance contracts • Grid abstract machine • Functionalities from resource, collective and connectivity • Middleware independent API Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Programming model • Component based • Inherits from existing ones (CCA, CCM, Proactive, …) • High performance • Builds on efficient mechanisms • Parallel • High level programming mechanisms and tools • Interoperability • Standard component frameworks • Code reuse • Sequential code in components Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Computational model: parallelism • Structured parallelism exploitation • Programmers quality of parallelism + contracts • Environment (compiler + run time) mechanisms, quantity, heuristics Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Computational model: interaction • Compositionality of components • Classic mechanisms: • RPC/RMI • Events • New mechanisms: • Data flow streams • Non deterministic control • Component reacts to incoming data Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Computational model: adaptivity • Performance contracts • Specified by the programmer • Performance requirements (bandwidth, parallelism degree, …) • Restructuring strategies • Specified by user • Program rewriting (different parallel patterns, parallelism degree, …) Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Comp. Model: adaptivity (2) • Allocation constrains • Mapping computations to nodes • Data access constrains • Adaptivity in the environment: • Monitoring of activities • Triggering of corrective actions • Notifying the user Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Non functional interface (contracts, adaptivity) Non functional interface (contracts, adaptivity) Existing code reuse Parallel structure Existing code reuse Parallel structure Seq. comp. Par. comp. legacy comp. New mechanisms (data flow streams) New mechanisms (data flow streams) Standard interface(uses/provides, events) Standard interface(uses/provides, events) Component model Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
ASSIST components GAM Application manager (AM) Abstraction of basic services Standard middleware Application manager • High performance components • ASSIST programs Grid Abstract Machine (GAM) Standard middleware Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
C0 C2 C4 C0 C1 C2 C4 C5 C1 C3 C5 C3 AM MAM1 MAM2 MAM3 MAM4 MAM5 AM MAM1 MAM2 MAM3 MAM4 MAM5 CAM01 CAM02 CAM0 CAM0 Application manager (2) Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Module Application Manager • Configuration control of single ASSIST parmod • Exploits Performance model repository • Classical models • History • Compiler or programmer responsibility • Adapts the parmod behaviour to satisfy module performance contract Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Performance Model Repository • Classical models • Parallelism degree = Compute time / Communication time • Dynamic models • Increase parallelism degree step-by-step until results start coming back • User provided • Proper formalism Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Component Application Manager • Global control strategies • Inputs from MAMs • Possibily overcomes MAMs proposals • E.g. 2 stage pipeline • First stage achieves performance contract ++ (very good resource recruited) • CAM requires new contract to the second stage Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Reconfiguration strategies • C1 virtualizes grid data resource in two streams • C2 is sequential by default strategy tells farm • C3 data parallelstrategy tells parallelism degree • C4 is sequential by default strategy tells if(Condx) data parallel else farm • C5 non det mapping of streams to data set • C2 bottleneck! • C2 new farm with proper parallelism degree • C4 C5 restructured as a consequence Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (1) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (2) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (3) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (4) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (5) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (6) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Experiments (7) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Conclusions • On going work • In the framework of GRID.it project • Involving major research institutions (grid, high performance, parallelism) in Italy • Main contribution • Non functional interfaces + application managers • Currently being experimented/implemented on/in ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004
Any questions ? di.unipi.it First.Family@ isti.cnr.it Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004