1 / 25

SciDAC TOPS PETSc Work

SciDAC TOPS PETSc Work. SciDAC TOPS Developers Satish Balay Chris Buschelman Matt Knepley Barry Smith Hong Zhang Other PETSc Team Members Lois Curfman McInnes, Bill Gropp, Dinesh Kaushik. http://www.mcs.anl.gov/petsc petsc-maint@mcs.anl.gov. PETSc.

lonna
Download Presentation

SciDAC TOPS PETSc Work

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SciDAC TOPS PETSc Work SciDAC TOPS Developers Satish Balay Chris Buschelman Matt Knepley Barry Smith Hong Zhang Other PETSc Team Members Lois Curfman McInnes, Bill Gropp, Dinesh Kaushik http://www.mcs.anl.gov/petsc petsc-maint@mcs.anl.gov

  2. PETSc • Portable, Extensible, Software for the scalable solution of nonlinear PDEs • Focusing on algebraic solvers • Preconditioners, multigrid, Krylov methods, Newton, FAS, time integrators

  3. Activities • Derivative calculations – ADIC/ADIFOR • Matrix-free preconditioners • FAS (future) • Accessing other software • HYPRE preconditioner usage • RAMG,SAMG • SuperLU • ESI • SIDL/Babel/CCA – PETSc 3.0 plans • SIDL/Babel MPI

  4. Derivatives • Automatic tools ADIC/ADIFOR are great but require manual processing (painful when developing code) • Would like to have a variety of derivatives “for free” (or nearly) • Matrix element based Newton which requires explicit sparse Jacobian • Matrix-free Newton • SOR preconditioners and FAS require one equation at at time

  5. To Fully Automate • Code comments to indicate what needs to be processed • Predefined calling sequences • Libraries that can use generated code • Make system automatically generates required derivative code

  6. typedef struct { double u,v,omega,temp; } Field; typedef struct { PassiveReal lidvelocity,prandtl,grashof; PetscTruth draw_contours; } AppCtx; .. /* Process adiC: FFL */ int FFL (DALocalInfo *info,Field **x, Field **f,void *ptr) { AppCtx *user = (AppCtx*)ptr; .. DMMGSetSNESLocal(dmmg,FFL,0,ad_FFL,admf_FFL);

  7. > make BOPT=g_c++ ex19 Running adiC on function FFL Extracting structure Field Extracting structure AppCtx Extracting function FFL adiC -a -d gradient -i /sandbox/bsmith/petsc/bmake/adic.init ex19.c.tmp ADIC 1.1 beta 4 08/07/00 18:06:31 This software was created in the course of academic and/or research endeavors by Argonne National Laboratory. See the disclaimer in the g++ -fPIC -o ex19.o -c -Dad_GRAD_MAX=36 -I/sandbox/bsmith/petsc/include/adic -g -I/sandbox/bsmith/petsc ..

  8. Matrix-free Multiply • Differencing (well understood) • AD typedef struct { double value;double grad[1]; } mfDERIV_TYPE; typedef struct { mfDERIV_TYPE u, v, omega, temp; } int mf_FFL(DALocalInfo *info,mf_Field **x, mf_Field **f,void *ptr) {

  9. Matrix-free Multigrid Preconditioners • Krylov smoothing with • no preconditioner • (point block) Jacobi preconditioner • (point block) SOR preconditioning

  10. Matrix-free Jacobi/SOR • If function is provided an “(block) equation at a time” matrix-free multiply can be done an equation at a time • can be computed by providing a seed of • For SOR • Similar computations with differencing

  11. FAS – Full Approximation Scheme (Nonlinear Multigrid) • Smoothing uses Newton to solve a single (block) equation at a time • Need the same “(block) equation at a time” and its derivative as for matrix-free Newton!

  12. We are developing a common software infrastructure and user interface for many Newton methods and FAS which can use AD or differencing for derivatives.

  13. Activities • Derivative calculations – ADIC/ADIFOR • Matrix-free preconditioners • FAS (future) • Accessing other software • HYPRE preconditioner usage • RAMG, SAMG • SuperLU • ESI • SIDL/Babel/CCA – PETSc 3.0 plans • SIDL/Babel MPI

  14. Philosophy on Solvers • Use any solver without changing the application code • Add to application code to take advantage of special solver features (optimizations) • but additions cannot break other solvers PCILUSetDropTolerance(pc,1.e-5); PCSPAISetEpsilon(pc,1.e-2);

  15. HYPRE-PETSc Matrix Element based preconditioners • No vector copies • Currently a matrix copy (plan to remove) -pc_type hypre -pc_hypre_type <pilut,parasails,boomerAMG,euclid> -pc_hypre_pilut_tol 1.e-4 -pc_hypre_parasails_thresh .1 -pc_hypre_parasails_filter .2

  16. RAMG, SAMG – PETScSequential Preconditioners • AMG – written by John Ruge, Klaus Stueben, and Rolf Hempel. A classic scalar PDE AMG code. • SAMG – written by Klaus Stueben. A complete rewrite that supports systems of PDEs (interface coming soon). • PETSc interfaces provided by Domenico Lahaye

  17. SuperLU_Dist • MPI parallel LU factorization and solve written by Sherry Li and Jim Demmel • Henry Tufo – Paul Fischer XYT fast direct parallel solver for “small problems”, i.e. the coarse grid problem in multigrid -mat_aij_superlu_dist -mat_aij_superlu_dist_colperm <MMD_AT_PLUS_A,NATURAL,MMD_ATA,COLAMD>

  18. ESI Equation Solver Interface • A C++ “standard” of abstract classes for setting up and solving linear systems • At least three implementations • Trilinos (Sandia, NM) • ISIS (Sandia, CA) • PETSc

  19. PETSc ESI Goal • Allow PETSc to use matrices and solvers written by others in ESI • Allow anyone to use PETSc matrices and solvers from ESI • Achieved by using two sets of wrappers • PETSc objects “look like” ESI objects • ESI objects “look like” PETSc objects

  20. PETSc using Trilinos objects -is_type esi –is_esi_type Petra_ESI_IndexSpace -vec_type esi -vec_esi_type Petra_ESI_Vector -mat_type esi –mat_esi_type Petra_ESI_CSR_Matrix ESI using PETSc objects esi::Vector<double,int> *v = new esi::petsc::Vector<double,int>(petscvector) esi::Operator<double,int> *o = new esi::petsc::Matrix<double,int>(petscmatrix)

  21. Activities • Derivative calculations – ADIC/ADIFOR • Matrix-free preconditioners • FAS (future) • Accessing other software • HYPRE preconditioner usage • RAMG, SAMG • SuperLU • ESI • SIDL/Babel/CCA – PETSc 3.0 plans • SIDL/Babel MPI

  22. SIDL/Babel (LLNL) • Scientific Interface Definition Language • Babel – generates language stubs for • C, C++, Fortran 77, Python, Java, … • Provides • Object model • Exception (error) model • Immature, but very exciting development

  23. PETSc 3.0 • SIDL/Babel for inter-language, object model and exception model • Multiple precisions • Single for preconditioners? • Vector and matrix sizes mutable • Adaptive mesh refinement • Not a rewrite, just a rewrap!

  24. Activities • Derivative calculations – ADIC/ADIFOR • Matrix-free preconditioners • FAS (future) • Accessing other software • HYPRE preconditioner usage • RAMG, SAMG • SuperLU • ESI • SIDL/Babel/CCA – PETSc 3.0 plans • SIDL/Babel MPI

  25. Conclusions • Many new tools and solvers • Accessible via • PETSc 2.0 interface • ESI • (coming) SIDL/Babel/CCA • We are happy to support users, especially those interested in nontrivial problems

More Related