A Computational Facility for Reacting Flow Science cfrfs.sandia

1. A Computational Facility for Reacting Flow Science cfrfs.ca.sandia.gov Abstract This project is focused on advancing capabilities for scientific studies of chemically reacting flow on massively parallel computational hardware. We are developing and using a flexible software toolkit for reacting flow computations, where distinct functionalities developed by experts are implemented in components in the context of the Common Component Architecture (CCA). We are doing this using a massively parallel C++ Adaptive Mesh Refinement framework, and are demonstrating the efficacy of the CCA approach by assembling components into multiple low Mach number codes, and participating in assembly of a compressible code by the SciDAC TSTC project. We are also developing and using advanced analysis/computation components that enable extraction of enhanced physical understanding of reacting flows from computational databases. These are based on the use of the Computational Singular Perturbation (CSP) technique for analysis of coupled transport-chemistry processes. We are implementing automatic chemical reduction with CSP and using it as the basis for an Adaptive Chemistry tabulation-based computational approach. We plan to demonstrate the assembly of these tools in computations of 2D/3D reacting flow, and to validate the results with respect to reacting flow databases. Assembly of CCA components In a Reaction-Diffusion Code Invariant manifolds in high-D chemical phase space CCA CSP+AMR-solver Assembly • Core CSP analysis implemented in CCA & runs online with reacting flow AMR solver • Flexible assembly allows CSP component reuse in alternate code and data analysis CCA constructs • online or offline • parallel Manifold Dimension Mirrors Flame Topology High Order AMR • Spatial discretizations of 1st & 2nd derivatives up to 8th order • Boundary conditions up to 12th order • Filters up to 12th order • Interpolants up to 8th order • Demonstrated up to 8th order spatial convergence for heat equation on 3 mesh levels CFRFS Team SNL - Habib Najm*, Jaideep Ray, Sophia Lefantzi, Jeremiah Lee, Christopher Kennedy1, Philippe Pebay U.Rome - Mauro Valorani*, Francesco Creta ICEHT - Dimitris Goussis UCD - Wolfgang Kollmann UCB - Michael Frenklach * Institutional Point of Contact 1 formerly at SNL Adaptive Mesh Topology: H2-O2 Ignition • Least number of exhausted modes, or highest dimensional manifold, observed in the reaction zone • Largest number of exhausted modes, or lowest dimensional manifold, observed in the products region, as the combustion products approach equilibrium Computational Cost Reacting Flow Code Challenges • Typical codes are inherently interconnected, forming an inseparable construction • Use of self-contained libraries is limited to specific solvers, e.g. for stiff integration, linear/nonlinear solvers. • Rearrangement of code sub-units to address different problems is often difficult, time consuming, and costly. • Code upgrades are essentially global upgrades • Code maintainability is a serious issue, particularly with regard to AMR and massively parallel implementations. T- Importance Index for Two Vortex Strengths • Rate of 4th order CPU increase is less than that of the 2nd order with stricter error tolerance. • For same error tolerance, 4th order requires less CPU Load • For same CPU Load get smaller errors with 4th order • The importance index measures influence of a process on time evolution of a state variable • Reveals cause-and-effect relationships • Importance index distributions for temperature are hardly changed between the two vortex cases • Similarly for other state variables H2-O2 Reaction-Diffusion Ignition CFRFS Reacting Flow Vision & Approach • Physical and Chemical complexity challenge • Need for detailed DNS studies of physical processes in combustion, including ignition, flame propagation, extinction, … • Target 3D parallel high order Adaptive Mesh Refinement computations of low Mach number reacting flow • Need for advanced chemical analysis & reduction tools • Develop and demonstrate Computational Singular Perturbation chemical analysis and reduction tools for 3D reacting flow – use for Adaptive Chemistry • Code and Data complexity (AMR+MPP+Chemistry) challenge • Need for advanced software architecture and collaboration with experts in other disciplines • Adopt CCA framework, work with CCTTSS & other ISICs • Need advanced analysis, data mining, & visualization tools • Develop CSP analysis toolkit, work w SDM & CMCS ISICs Scalability • Intel/Myrinet NCSA cluster • 2D H2-O2 ignition, 100x100 base mesh, 4 mesh levels • Space Filling Curve load balancer • Implicit chemistry integration involves more CPU work per mesh cell per time step in the primary flame region - Adversely affects scalability - Specialized chemistry load balancing is necessary Distribution of CSP Radicals on Vortex-Pair Centerline • CSP radicals are the optimal species to be found from linear algebraic relationships with time-integrated species • Their identification provides the corresponding simplified chemical models at different spatiotemporal locations • Their distribution exhibits minor dependence on the vortex-pair strain-rate CFRFS Plans • Demonstrated high-order spatial convergence with CCA GrACE/AMR • Demonstrated 2nd-order operator-split computations with stiff CVODE integration • Demonstrated 2D high-order AMR reaction diffusion computations with detailed hydrogen chemistry and transport • Model problem: random-kernel heat-source premixed H2-O2 ignition, kernel growth, and flame propagation CFRFS Progress • Computational • Low Mach number AMR momentum solver (w/ APDEC) • Operator-split AMR momentum+species solver • Parallel scalability and dynamic load balancing • 3D AMR/CCA low Mach number reacting jet flow • GrACE/CCA TSTC support • Analysis • PRISM tabulation of CSP exhausted vectors • Adaptive PRISM-CSP Adaptive chemistry time integration • CSP analysis of general stochastic dynamical systems • CSP analysis and reduction of complex fuel chemistry • GrACE/CCA/CSP data analysis toolkit Goals • Develop a 3D flexible massively parallel CCA-based reacting flow computation and analysis high-order AMR code toolkit • Demonstrate Adaptive Chemistry AMR computations and chemical analysis and reduction in multi-dimensional reacting flow • Computational • Mesh, Thermo-Chemistry & Transport CCA components • High-order AMR 3D spatial derivatives & interpolants comps. • Operator-split CVODE-RKC AMR time integration comps. • Demonstrated 2D high-order AMR/CCA reaction-diffusion • Analysis • CSP analysis & reduction kernel CCA components • Improved CSP analysis robustness using SVD • Analysis of 2D GRImech3.0 flame-vortex data • Demonstrated slow spatial variability of exhausted CSP vectors --- consequence to feasibility of tabulation • Demonstrated fast explicit CSP+RK integration of model stiff reaction-diffusion system SciDAC Collaborations Impact • Enable facile use of large-scale supercomputing platforms for advanced reacting flow studies and scientific discovery • Improved understanding of fundamental reacting flow • Enable the development of predictive combustion models • CCTTSS - CCA Software framework • APDEC - Poisson solution for momentum solver • TSTC - Compressible DNS implementation • CMCS - Visualization, Feature-tracking • SDM - Data mining, Feature-tracking SciDAC PI Meeting, Charleston, SC March 22-24, 2004