Application of Design Patterns to Geometric Decompositions

1. Application of Design Patterns to Geometric Decompositions V. Balaji, Thomas L. Clune, Robert W. Numrich and Brice T. Womack

2. Our prototype example A grid-based code • implements finite-difference operators over fields defined on a grid of coordinates representing physical space. Solution of these problems typically involve: • decomposition of data • distribution of data across a parallel architecture • assignment of work to that data.

3. A typical Approach Partition the physical coordinate space into subdomains. • subdomains are extended to include regions of overlap with neighbors. • i.e. - halo cells, ghost cells, or guard cells • subdomains interact with each other in some manner so they know when to update their halo cells with fresh data from their neighbors. • The size, number and distribution of subdomains needs to be balanced to keep each processor busy and allow efficient exchange of halo data

4. Gof Patterns Applied Here Our prototype uses the following OO design patterns to solve a geometric decomposition problem • Composite • Builder • Strategy • Mediator • Iterator • …

5. GoF Composite Pattern

6. GoF Composite Pattern • Allows us to maintain a unified picture of subdomains whether we want to think of them in: • a global setting, the problem as a whole • a local setting, the problem as local pieces. • Methods associated with the composite provide a common interface at each level of the decomposition and allow the software developer to move from one view of the problem to another in a consistent manner.

7. Composite Tree Structure Comp Map Shared Memory Node Group 1 Shared Memory Node Group 2 Shared Memory Node Group …N P1 P1 P2 P3 P1 P4 P5 P1 P1 P6 P… P1 L1 L4 L7 L10 L13 L16 L… L2 L5 L8 L11 L14 L17 L… L3 L6 L9 L12 L15 L18 L…

8. Composite features • Application code is written to expect a block of data from a leaf. It shouldn’t assume any specific : • index order • data layout • or decomposition • Blocks of data are assigned to processors by the object map structure • migration of work is done by simply moving one or more leaves to a different location within the composite • Pieces of work (leaves) can be adaptively refined(AMR splitting/combining of sub elements) by replacing a leaf with a composite containing the newly refined leaves or the reverse operation

9. GoF Builder Pattern • By applying the Builder pattern, we can isolate the composite's creation steps to produce composites that are: • Optimal for a target compute architecture • Customized for a particular numerical algorithm • Adapted for a particular set of physical or geometric constraints. • Etc.

10. GoF Builder Pattern

11. GoF Builder with Strategies

12. GoF Strategy Pattern • Using the Strategy Pattern, we can provide a set of standard ways to build our composite object maps • Users can select from a set of predefined build strategies. • Users can easily customize the object map decomposition to improve performance. • Allows easy testing of application performance using different object map configurations

13. GoF Composite with Mediators

14. Data exchange w/Mediators • Each local leaf uses a Mediator that is configured by the builder to manage its communication with neighbors • All halo regions are specified in the local leaf’s context, not the neighbors • A local halo region can transparently map to multiple neighbors • the user’s code doesn’t see any difference between 1 neighbor or 10 neighbors • Mediator’s can use different communication methods for neighbors that are in local memory and neighbors that are outside of local memory

15. GoF Composite Applied

16. Request, Require & Release • Request – non-blocking notification asking for data from neighbors • Require – blocking request for data; does not continue until fulfilled • Release – non-blocking notification of completion of work on local data • Depending on the specific build strategy used to create the composite, the Mediator’s Request, Require and Release methods may be implemented using: • MPI or MPI-2 • SHMEM • Co-Array Fortran • ARMCI, • or Threads • By abstracting the parallel communication mechanism, the builder has considerable freedom in configuring each Mediator to efficiently map communication implementation to the specific target parallel architecture.

17. Gof Iterators • In a typical grid-code, each processor works only on the set of subdomains that are located in its own local memory. • However, we also want to enable the use of global task queues for software architectures that permit low-latency task-switching when tasks stall for memory. • We therefore see a need to provide iterators that allow: • Local iteration through all subdomains within local memory independent of the specific composition or the specific strategy used to build the composition. • Global iteration over subdomains that do not reside in the local memory of the physical processor doing the work.

18. Conclusions • We are still working out implementation details, but, we have prototype Java code that is performing shallow water model equations based on these Gof Design patterns • We are finding that application of the GoF design patterns can add flexibility and reduce the complexity of managing geometric decomposition problems • We intend to use Fortran 2003’s OO features to implement this design when compilers are ready