1 / 20

Using Communication Patterns for Parallel Program Optimization

Using Communication Patterns for Parallel Program Optimization. Robert Preissl 1,2 , Dieter Kranzlmüller 1 , Martin Schulz 2 , Bronis de Supinski 2 , Dan Quinlan 2 1 GUP , Joh. Kepler Univ. Linz, Austria 2 CASC , Lawrence Livermore Natl. Lab, US. Output. Program. Input. Output. Input.

amity-cain
Download Presentation

Using Communication Patterns for Parallel Program Optimization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Communication Patterns for Parallel Program Optimization Robert Preissl1,2, Dieter Kranzlmüller1,Martin Schulz2, Bronis de Supinski2, Dan Quinlan2 1 GUP, Joh. Kepler Univ. Linz, Austria2 CASC, Lawrence Livermore Natl. Lab, US

  2. Output Program Input Output Input Program Analysis What is the program „really“ doing? Using Communication Patterns ...

  3. Parallel and DistributedPrograms Output Input Using Communication Patterns ...

  4. Parallel Program Analysis Using Communication Patterns ... VAMPIR (TU Dresden)

  5. ParallelProgramAnalysis Using Communication Patterns ... ATEMPT (GUP Linz)

  6. Scalability: # of Proc‘s? ParallelProgramAnalysis Using Communication Patterns ...

  7. Scalability:Number of Processes? Using Communication Patterns ...

  8. Scalability: Time? ParallelProgramAnalysis Using Communication Patterns ...

  9. Loop Detection Check correctness of iterative patterns Using Communication Patterns ...

  10. Almost Correct Patterns Using Communication Patterns ...

  11. Pattern Detection Algorithm • Extraction of communication patterns from MPI Traces • Usually applied for error detection  Can it be used for performance optimization? • Combine dynamic analysis with static source code analysis Using Communication Patterns ...

  12. Combining Dynamic Patterns with Static Source Code Analysis • Pattern Detection Algorithm • Associate Patterns with Nodes in AST • Static Analysis • Source Code Transformation Using Communication Patterns ...

  13. AST Traversal C++ Front-End Unoptimized C++ Source C++ AST C++ AST Transformation AST Rewrite System Rewritten C++ AST Optimized C++ AST C++ Code Generator Optimized C++ Source Data Optimized Executable Vendor C++ Compiler Uses The ROSE Compiler Framework • Open Source Infrastructure for Source-to-Source Translation • Provides Mechanisms for reading, analyzing and rewriting C/C++ codes Using Communication Patterns ...

  14. The ROSE Abstract Syntax Tree ROSE AST Features: • AST Query mechanisms • AST Rewrite mechanisms • Semantic actions associated with grammar rules • Abstract C++ grammar is predefined • Higher level grammars automatically generated from library source • Source code generation Using Communication Patterns ...

  15. Static Analysis: SDG(System Dependence Graph) • Directly labeled multigraph capturing control & data dependencies Using Communication Patterns ...

  16. Inefficient communication structures: “Master-Slave Broadcast“ Unoptimized C++ Source: Identify MPI_Send, MPI_Recv in AST of source code for (nr_repeats) { if( MASTER ) { for (nr_slaves) { MPI_Send( SLAVE ) } } if( SLAVE ) { MPI_Recv(MASTER) } } Optimized C++ Source: for (nr_repeats) { MPI_Bcast() } Source Code Transformation Using Communication Patterns ...

  17. Inefficient communication structures: “Hypercube Broadcast“ Unoptimized C++ Source: for (nr_repeats) { for (nr_levels) { if( Send( &SendID ) ) { MPI_Send( SendID ) } elseif( Recv( &RecvID ) ) { MPI_Recv( RecvID ) } } Send_Data_To_Master () } Identify MPI_Send, MPI_Recv in AST of source code Optimized C++ Source: for (nr_repeats) { MPI_Bcast () Send_Data_To_Master () } Source Code Transformation Using Communication Patterns ...

  18. Inefficient communication structures: “Late Sender“ Optimized C++ Source: MPI_Request req Data x, y for (nr_repeats) { if( task0 ) { Do_Computation ( x ) MPI_Send ( task1 , x ) } if( task1 ) { MPI_Irecv( task0 , x, &req) } if( task1 ) { Do_Computation ( y ) } if( task1 ) { MPI_Wait ( &req ) Do_Computation ( x ) } } Unoptimized C++ Source: Data x, y for (nr_repeats) { if( task0 ) { Do_Computation ( x ) MPI_Send ( task1 , x ) } if( task1 ) { MPI_Recv( task0 , x) } if( task1 ) { Do_Computation ( y ) } if( task1 ) { Do_Computation ( x ) } } Identify MPI_Recv in AST Source Code Transformation Using Communication Patterns ...

  19. Overview of Approach Using Communication Patterns ...

  20. Summary • Pattern matching offers • further abstraction of event graphs to cope with large-scale traces • detection of incorrect communication structures • hints for optimization of program performance • Automatic verification of patterns relieves users during program analysis and improves comprehension of program behavior Using Communication Patterns ...

More Related