1 / 15

Adaptive MPI: Flexible Parallel Programming for Dynamic Applications

Adaptive MPI is designed for highly dynamic parallel applications to address issues such as load imbalance and programming complexity. With features like virtual processors and automatic load balancing, it offers a seamless parallel programming experience on various computing platforms. This innovative approach allows for adaptive overlapping between computation and communication, enhancing performance and scalability. Ongoing work includes automatic checkpointing and performance prediction for robustness and optimization.

derekbryant
Download Presentation

Adaptive MPI: Flexible Parallel Programming for Dynamic Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adaptive MPI Chao Huang Parallel Programming Lab UIUC

  2. Motivation • Issues • Highly dynamic parallel applications • Usually limited supercomputing platforms availability • Load imbalance and programming complexity • Little change to standard MPI program • Virtual processors • +vp option allows execution on desired number of virtual processors • Adaptive overlapping between computation and communication • Automatic load balancing Adaptive MPI

  3. Outline • Motivation • Implementation • Features • Ongoing Work • Future Work Adaptive MPI

  4. MPI processes MPI “processes” Implemented as virtual processes (user-level migratable threads) Real Processors AMPI: MPI with Virtualization • Each virtual process implemented as a user-level thread associated with a message-driven object Adaptive MPI

  5. Virtualization • Basic idea • Virtual MPI processors mapped onto physical processors • Typically, # virtual processors > # processors • Advantages • Run program on any given number of processors • Adaptive overlapping computation and communication • Mapping strategy helps load balancing Adaptive MPI

  6. Adaptive Overlapping Problem setup: Jacobi 3D problem size 2403 run on LeMieux. Run with virtualization ration 1 and 8. (p=8, vp=8 and 64) Adaptive MPI

  7. Speedup Problem setup: Jacobi 3D problem size 2403 run on LeMieux. Shows AMPI with virtualization ratio of 1 and 8. Adaptive MPI

  8. Virtualization Problem setup: Jacobi 3D problem size 2403 run on LeMieux. AMPI runs on any given # of PEs (eg 19, 33, 105), but native MPI needs cube #. Adaptive MPI

  9. Load Balancing • Dynamic load balancing • Maps and re-maps objects as needed • Re-mapping strategies help adapt to dynamic variations • Load balancing by object migration: MPI_Migrate() • Collective call informing the load balancer that the thread is ready to be migrated, if needed • The load balancer migrates the objects • Packing, transferring and Unpacking (PUP) Adaptive MPI

  10. Load Balancing Example AMR application Load balancer is activated at time steps 20, 40, 60, and 80. Adaptive MPI

  11. Asynchronous Communications • Collective communications in MPI are complex and time consuming • MPI_Alltoall, etc • Implemented as blocking calls in MPI • Asynchronous calls enable overlapping computation with communication • Powered by communication optimization library MPI_Ialltoall(...) /* Some computation here */ MPI_Waitall(...) Adaptive MPI

  12. Communication Optimization Alltoall time on 1K processors Adaptive MPI

  13. Communication Optimization Alltoall CPU Overhead on 1K processors Adaptive MPI

  14. Ongoing work • Automatic checkpointing • Improve robustness of large-scale applications • Performance prediction via direct simulation • Facilitate performance tuning w/o continuous access to large machine Adaptive MPI

  15. Future Work • Support for visualization • Complete compliance to MPI-1.1 • Support for MPI-2 standard Adaptive MPI

More Related