Adaptive grid refinement

1. Adaptive grid refinement

2. Adaptivity in Diffpack • Error estimator • Adaptive refinement • A hierarchy of unstructured grids • Solution method: Multigrid, often optimal complexity • As an add-on Diffpack library

3. Parallel Computing in Diffpack

4. Two parallelization approaches • Flexible, user-friendly, extensible • Linear-algebra-level parallelization • An add-on library of parallel matrix/vector operations • Rapid transition from sequential simulator to parallel simulator • Simulator-parallel approach • High-level parallelization of multilevel methods • Extensive reuse of original sequential simulator in subdomain solves • A generic implementation framework

5. Grid partition • Basis for computation work decomposition • General treatment of unstructured grids • Arbitrary number of procs determined at run-time • Non-overlapping partition • Controllable addition of overlap (if desired) • Good load balancing - high parallel efficiency

6. Partition example

7. Linear-algebra-level parallelization • Easy to use • access to all existing Diffpack iterative methods, preconditioners and convergence monitors • need only to add a few lines of new code • arbitrary choice of number of procs at run-time • Parallel matrix/vector operations • inner-product of two vectors • matrix-vector product • preconditioning - block contribution from subgrids

8. Example one Highly unstructured grid Discontinuity in the coefficient

9. Measurements for example one 130,561 degrees of freedom Overlapping subgrids BiCGStab + (block) RILU prec.

10. An observation “The hope among early domain decomposition workers was that one could write a simple controlling program which would call the old PDE software directly to perform the subdomain solves. This turned out to be unrealistic because most PDE packages are too rigid and inflexible.” - Smith, Bjørstad and Gropp The remedy: Correct use of object-oriented programming techniques.

11. Approach 2: simulator-parallel • Parallelization of multilevel methods • One subdomain is assigned with a sequential simulator • A generic implementation framework • Systematic and flexible • O-O programming enables extensive code reuse • Easy to incorporate multilevel algorithm modification • Different grid types, local solution methods etc. on different subdomains • DD on non-matching subgrids

12. Example two Laplace equation

13. Measurements for example two BiCGStab - global iterative solution method Parallel DD as preconditioner MG as subdomain solver Overlapping subgrids

14. Measurements for example two BiCGStab - global iterative solution method One parallel MG iteration as preconditioner Overlapping subgrids

15. Scalability Measurements obtained on 16 processors

16. Example three

17. Measurements for example three BiCGStab - global iterative solution method One parallel MG iteration as preconditioner Overlapping subgrids

18. 2-phase porous media flow

19. 2-phase porous media flow SEQ: PEQ: Multigrid V-cycle in subdomain solves

20. Nonlinear water waves Fully nonlinear 3D water waves Primary unknowns:

21. Parallel simulation of water waves

22. Parallel efficiency • Fixed number of subdomains M=16. • Subdomain grids from partition of a global 41x41x41 grid. • Simulation over 32 time steps. • DD as preconditioner of CG for the Laplace eq. • Multigrid V-cycle as subdomain solver.

23. Overall efficiency • Number of subdomains equal to number of processors *ForP=2 parallel BiCGStab is used.