1 / 73

The P-GRADE Visual Parallel Programming Environment

The P-GRADE Visual Parallel Programming Environment. Péter Kacsuk Laboratory of Parallel and Distributed Systems MTA SZTAKI Research Institute kacsuk@sztaki.hu www.lpds.sztaki.hu. Programming?. Observing?. Problems of Developing Parallel Programs. High-Speed Switch.

moanna
Download Presentation

The P-GRADE Visual Parallel Programming Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The P-GRADE Visual Parallel Programming Environment Péter Kacsuk Laboratory of Parallel and Distributed Systems MTA SZTAKI Research Institute kacsuk@sztaki.hu www.lpds.sztaki.hu

  2. Programming? Observing? Problems of Developing Parallel Programs High-Speed Switch

  3. Our Solution: P-GRADE • P-GRADE is a parallel programming environment which supports the whole life-cycle of parallel program development • For non-specialist programmers it provides a complete solution for efficient and easy parallel program development • Fast reengineering of sequential programs for parallel computers • Unified graphical support in program design, debugging and performance analysis • Portability on supercomputers and heterogeneous workstation/PC clusters based on PVM and MPI

  4. Tools of P-GRADE • GRAPNEL: Hybrid Parallel Prog. Language • Graphics to express parallelism • C/C++ to describe sequential parts • GRED: Graphical Editor • GRP2C: Pre-compiler to (C/C++)+(PVM/MPI) • DIWIDE: Integrated distributed debugger and animation system • GRM: distributed monitoring system • PROVE: Integrated visualisation tool

  5. Parallel Program Design GRAPNEL GRED GRP file Mapping User mapping GRP file Pre-compilation PVM Library MPI Library GRP2C C source code, Cross-ref file, Make file GRM Library GRM Library Building executables GRP-PVM C compiler, linker GRP-MPI executables Debugging Monitoring DIWIDE GRM Trace file Visualisation PROVE Life-cycle of Parallel Program Development and its support in P-GRADE

  6. Design Goals of GRAPNEL • Graphical interface • to define all parallel activities • Strong support for hierarchical design • Visual abstractions to hide the low level details of message-passing • C/C++ (or Fortran) to describe sequential parts • Strong support for parallelizing sequential applications • Support for programming in large • No steep learning curve • GRAPNEL = (C/C++) + graphics

  7. GRAPNEL: GRaphical Process NEt Language • Programming paradigm: message-passing • component processes run in parallel and can interact only by means of sending and receiving messages • Communication model: • point-to-point, synchronous/asynchronous • collective (e.g. multicast, scatter, reduce, etc.) • Process model: • single processes • process groups • predefined process communication templates

  8. Three layers of GRAPNEL

  9. GRAPNEL • Hierarchical design levels: • Graphics used at application level: • Defines interprocess communication topology • Port protocols • Graphics hides PVM/MPI function calls • Support for SPMD programming style • Predefined communication patterns • Automatic scaling of parallel programs

  10. Communication Templates • Pre-defined regular process topologies • process farm • pipeline • 2D mesh • tree • User defines: • representative processes • actual size • Automatic scaling

  11. Mesh Template

  12. Tree Template

  13. spawn(N); Master Send work packages send(); Slave1 Slave2 SlaveN recv(); Collect results The process farm parallelisation approach The code of each slave is the same.

  14. Parallelising the Mandelbrot set computation

  15. Draw process output

  16. Compute process input

  17. Compute process output

  18. Draw process input

  19. Process Groups • Hierarchical design(subgraph abstraction) • Collective communication(group ports) • multicast • scatter • gather • reduce

  20. GRAPNEL • Hierarchical design levels: • Graphics used at process internal level • C/C++ used at the text level • Synch/asynch. comm. • Programming in large: • Any C/C++ library call can be included in text blocks • Graphical support for object-based programming

  21. GRAPNEL • Structuring facility by macro graphs

  22. multicast scatter gather GRAPNEL reduce Userdef (grp_in) Point-point Userdef (grp_out) gather scatter

  23. Parallelising the Mandelbrot set computation

  24. Parallelising the Mandelbrot set computation

  25. Parallelising the Mandelbrot set computation

  26. Parallelising the Mandelbrot set computation

  27. GRED Editor • Supports the creation of all the elements of GRAPNEL • Drag-and-drop style of drawing • Cut/copy/paste/move on graphical objects • Automatic port positioning with minimal lengths and crossing of communication channels

  28. GRED Editor • Extremely easy and fast construction of process graph • Automatic arrange of the process graph • Automatic resizing of process windows • Cut/copy/paste on graphical objects • Macro graph construction at arbitrarily nested level • C/C++ code can be edited by any standard text editor

  29. GRAPNEL Generated code C/C++ C/C++ graphics GRP2C PVM/MPI GRP2C Pre-compiler • Automatic generation of PVM and MPI calls based on GRAPNEL graphics • Automatic code instrumentation for debugging and performance monitoring

  30. Observing? Debugging Parallel Programs High-Speed Switch

  31. Principle of sequential program debugging • Reproducibility - determinism • For the same input set the sequential program delivers always the same output set (even if the program is incorrect) • Used technique: cyclic debugging • breakpoints • step-by-step execution

  32. Problem of parallel program debugging • Non-reproducibility (non-determinism) • For the same input set the incorrect parallel program can deliver different output sets • Cyclic debugging cannot be used • breakpoints • step-by-step execution

  33. Classification of parallel debuggers Parallel running seq. debuggers Replayable debuggers Monitor&replay Control&replay

  34. DIWIDEDebugger • Graphical and C/C++ level debug support (breakpoints, variable inspection, etc.) • 3 kinds of “step by step execution”, according to the programmer’s demand: • Instruction by instruction, • Graphical item by graphical item, • Macrostep by macrostep • Visualisation and animation support

  35. Hierarchical Debugging by DIWIDE

  36. Classification of parallel debuggers Parallel running seq. debuggers Replayable debuggers Monitor&replay Control&replay

  37. Classification of parallel breakpoints Local breakpoints Global breakpoints Individual breakpoints Collective breakpoints

  38. P2 P3 P1 S2 S3 S1 A3 A1 A2 B2 C2 B3 B1 D2 E3 E1 E2 Principle of Macrostep Debugging Macrosteps Collective Breakpoints M0 = {S1-> A1, S2-> A2, S3-> A3} A1 A2 A3 M1 = {A1-> B1, A2-> B2, A3-> B3} B1 B2 B3 M2 = {B1-> B1, B2-> C2, B3-> B3} B1 C2 B3 M3 = {B1-> B1, C2-> D2, B3-> E3} B1 D2 E3 M4 = {B1-> E1, D2-> E2} E1 E2 where Si = Starti and Ei = Endi • Parallel debugging is as easy as debugging traditional sequential programs.

  39. Support for systematic debugging to handle non-deterministic behaviour of parallel applications Systematic and automatic generation of Execution Trees Testing parallel programs for all time conditions Replay technique with collective breakpoints Macrostep Debugging

  40. Automatic Deadlock Detection by Macrostep Debugging

  41. Integration of Macrostep Debugging and PROVE

  42. Observing? Performance monitoring and analysis of Parallel Programs High-Speed Switch

  43. Visualisation Systems What to visualise? Scientific (Data Oriented) Visualisation Program Visualisation Problem Visualisation (Alg. Animation) Goal of visualisation? Correctness Debugging Performance (Debugging) Visualisation Combined Visualisation

  44. Program Visualisation Goal of visualisation? Correctness Debugging Performance Visualisation Combined Visualisation When to visualise? On-line Off-line Semi On-line

  45. Phases of Performance Visualisation Source Code Instrumentation (GRAPNEL/GRED) Runtime Monitoring (GRM) Visualisation (PROVE) Data Analysis (PROVE)

  46. Performance Visualisation Evaluation Criteria Source Code Instrumentation Scalability (Data handling) Versatility (Visualisation)

  47. Source Code Instrumentation Manual or Automatic Filtering Monitoring modes Click-back facility Selectable program units On/off facility Individual Events Statistics

  48. Scalability Data Acquisition Data Analysis & Display Filtering Turning tracing on/off Zooming Filtering Interactive Non-Interactive VISTOP Nupshot

  49. Versatility Different views Interoperate with other tools Event views Statistics views No Yes

  50. Standalone Performance Analysis Tools • VAMPIR • Pablo • ParaGraph • AIMS • Paradyn

More Related