Computational Framework for Subsurface Energy and Environmental Modeling and Simulation

1. Computational Framework for Subsurface Energy and Environmental Modeling and Simulation Mary Fanett Wheeler, Sunil Thomas Center for Subsurface Modeling Institute for Computation Engineering and Sciences The University of Texas at Austin

2. Acknowledge Collaborators: Algorithms: UT-Austin (T. Arbogast, M. Balhoff, M. Delshad, E. Gildin, G. Pencheva, S. Thomas, T. Wildey): Pitt (I. Yotov); ConocoPhillips (H. Klie) Parallel Computation: IBM (K. Jordan, A. Zekulin, J. Sexton); Rutgers (M. Parashar) Closed Loop Optimization: NI (Igor Alvarado, Darren Schmidt) Support of Projects: NSF, DOE, and Industrial Affiliates (Aramco, BP, Chevron, ConocoPhillips, ExxonMobil, IBM, KAUST)

3. Outline Introduction General Parallel Framework for Modeling Flow, Chemistry, and Mechanics (IPARS) Solvers Discretizations Multiscale and Uncertainty Quantification Closed Loop Optimization Formulations (IPARS-C02) Compositional and Thermal Computational Results Validation and Benchmark Problems Current and Future Work

4. Societal Needs in Relation to Geological Systems Resources Recovery • Petroleum and natural gas recovery from conventional/unconventional reservoirs • In situ mining • Hot dry rock/enhanced geothermal systems • Potable water supply • Mining hydrology • Site Restoration • • Aquifer remediation • Acid-rock drainage Waste Containment/Disposal • Deep waste injection • Nuclear waste disposal • CO2 sequestration • Cryogenic storage/petroleum/gas Underground Construction • Civil infrastructure • Underground space • Secure structures

5. Highly Integrated Multidisciplinary, Multiscale, Multiprocess Scientists and Engineers in Geotechnology Physics Mathematics Exploration Characterization Diagnostics Geophysics Geological Eng. Geology Exploration Characterization Earth Stresses Mechanical Rock/Soil Behavior Geomechanics Geomechanics GeoHydrology Fluid Flow Waste isolation Petroleum Engineering Hydrocarbon Recovery Simulation Construction Soil Mech. Rock Mech. Struct. Anal. Civil Engineering Geochemistry Mining Engineering Mining Design/ Stab, Waste, Land Reclamation Waste isolation Computer Sciences Mechanical Engineering Drilling & Excav., Support, Instruments Code Development, Software Engineering

6. Long Range Vision: Characterization And Management of DiverseGeosystems Sensor Data Management Uncertainty Assessment Characterization & Imaging Sensor Placement 3D Visualization & Interpretation Complex Geosystem Management Optimization and Control Data Management Multiscale Simulation Geophysical Interpretation Petrophysical Interpretation Multiphysics Simulation A Powerful Problem Solving Environment

7. Framework Components High fidelity algorithms for treating relevant physics: Locally Conservative discretizations (e.g. mixed finite element and DG) Multiscale (spatial & temporal multiple scales) Multiphysics (Darcy flow, biogeochemistry, geomechanics) Complex Nonlinear Systems (coupled near hyperbolic & parabolic/ elliptic systems with possible discrete models) Robust Efficient Physics-based Solvers (ESSENTIAL) A Posteriori Error Estimators Closed loop optimization and parameter estimation Parameter Estimoation (history matching) and uncertainty quantification Computationally intense: Distributed computing Dynamic steering

8. The Instrumented Oil Field Detect and track changes in data during production. Invert data for reservoir properties. Detect and track reservoir changes. Assimilate data & reservoir properties into the evolving reservoir model. Use simulation and optimization to guide future production.

9. IPARS: Integrated Parallel and Accurate Reservoir Simulator

10. PHYSICS BASED SOLVERS K Tensor Fractures Physics Flow Regimes Heterogeneity Multiple Physics Well Operations Numerical representation Insights AMG FDM MFE AML HPC Discretization Solvers CVM DD DG Numerical Solution Krylov MPFA LU/ILU Mortar Soft Computing Reinforced Learning Random Graph Theory Multiresolution Analysis Randomized Algorithms Physics-based Solvers

11. Why Multiscale? Upscale • Subsurface properties vary on the scale of millimeters • Computational grids can be refined to the scale of meters or kilometers • Multiscale methods are designed to allow fine scale features to impact a coarse scale solution • Variational multiscale finite elements • Hughes et al 1998 • Hou, Wu 1997 • Efendiev, Hou, Ginting et al 2004 • Mixed multiscale finite elements • Arbogast 2002 • Aarnes 2004 • Mortar multiscale finite elements • Arbogast, Pencheva, Wheeler, Yotov 2004 • Yotov, Ganis 2008

12. Basic Idea of the Multiscale Mixed Mortar Method

13. Multiscale Mortar Mixed Finite Element Method

14. Domain Decomposition and Multiscale Domain Decomposition Multiscale Approach For each stochastic realization, time step and linearization For each stochastic realization, time step and linearization Compute data for interface problem Compute data for interface problem Subdomain solves Subdomain solves Multiple subdomain solves Precondition data Compute multiscale basis for coarse scale Apply precond. Multiple subdomain solves Multiple linear combinations of basis Solve the interface problem Solve the interface problem Multiple precond. applications Solve local problems given interface values Solve local problems given interface values Subdomain solves Subdomain solves

15. Domain Decomposition and Multiscale Multiple subdomain solves Compute the multiscale basis for a training operator For each stochastic realization, time step and linearization Subdomain solves Compute data for interface problem Apply Multiscale precond. Precondition data Fixed number of subdomain solves Solve the interface problem Fixed number of multiscaleprecond. applications Solve local problems given interface values Subdomain solves

16. Example: Uncertainty Quantification • 360x360 grid • 25 subdomains of equal size • 129,600 degrees of freedom • Continuous quadratic mortars • Karhunen-Loéve expansion of the permeability truncated at 9 terms • Second order stochastic collocation • 512 realizations • Training operator based on mean permeability Mean Permeability Number of Interface Iterations Interface Solver Time Mean Pressure

17. Example: IMPES for Two Phase Flow • 360x360 grid • 25 subdomains of equal size • 129,600 degrees of freedom • Continuous quadratic mortars • 50 implicit pressure solves • 100 explicit saturation time steps per pressure solve • Training operator based on initial saturation Absolute Permeability Number of Interface Iterations Initial Saturation Interface Solver Time

18. Finite Element Oxbow Problem

19. FD & FEM Couplings: 3 Blocks with Fault

20. Solution

21. Continuous Measurement and Data Analysis for Reservoir Model Estimation Source: E. Gildin, CSM, UT-Austin

22. Continuous Measurement and Data Analysis for Reservoir Model Estimation LabVIEW Optimization & Supervisory Control Field Controller(s) IPARS Reservoir Dynamic I/F LabVIEW Online Analysis (Data Fusion, Denoising, Resampling…) Data Assimilation (EnKF) Data Acquisition (Sensors + DAQ) Source: I. Alvarado and D. Schmidt, NI

23. Parameter Estimation Using SPSA

24. Key Issues in C02 Storage What is the likelihood and magnitude of CO2 leakage and what are the environmental impacts? How effective are different CO2 trapping mechanisms? What physical, geochemical, and geomechanical processes are important for the next few centuries and how these processes impact the storage efficacy and security? What are the necessary models and modeling capabilities to assess the fate of injected CO2? What are the computational needs and capabilities to address these issues? How these tools can be made useful and accessible to regulators and industry? drinking-water aquifer groundwater flow CO2 leakage deep brine aquifer

25. Global Experience in CO2 Injection From Peter Cook, CO2CRC

26. CO2 Sequestration Modeling Approach Numerical simulation Characterization (fault, fractures) Appropriate gridding Compositional EOS Parallel computing capability Key processes CO2/brine mass transfer Multiphase flow During injection (pressure driven) After injection (gravity driven) Geochemical reactions Geomechanical modeling

27. IPARS-COMP Gridding Parallel Solvers Geochemical Reaction EOSComp. Geomechanics Thermal 2-P Flash Graphics Physical Prop Numerics

28. IPARS-COMP Flow Equations Mass Balance Equation Pressure Equation Solution Method Iteratively coupled until a volume balance convergence criterion is met or a maximum number of iterations exceeded.

29. Thermal & Chemistry Equations Energy Balance Solved using a time-split scheme (operator splitting) Higher-order Godunov for advection Fully implicit/explicit in time and Mixed FEM in space for thermal conduction Chemistry System of (non-linear) ODEs Solved using a higher order integration schemes such as Runge-Kutta methods

30. Coupled Flow-Thermal-Chemistry Algorithm

31. CO2 EOR Simulations

32. Validation SPE5 -- A quarter of 5 spot benchmark WAG problem 3-phase, 6 components C1, C3, C6, C10, C15, C20 IPARS-CO2 vs CMG-GEM Cum. oil produced Cum. gas Inj Prod

33. Validation CO2 pattern flood injection 3-phase, 10 components CO2, N2, C1, C3, C4, C5, C6, C15, C20 IPARS-CO2 vs CMG-GEM CO2 conc. Cum. gas Inj Prod.

34. Parallel Simulations Modified SPE5 WAG injection Permeability from SPE10 160x160x40 (1,024,000 cells) 32, 64, 128, 256, 512 processors Oil pressure and water saturation @ 3 yrs Gas saturation and propane conc. @ 3 yrs

35. Parallel Scalability Texas Advanced Computing Center The University of Texas at Austin

36. Scalability On Ranger (TACC) & Blue Gene P GMRES solver with Multigrid Preconditioner 3500ft, 3500 ft, 100ft reservoir 40x160x160=1,024,000 elements CPUs: 32, 64, 128, 256, 512, 1024 Ranger (TACC) Blue Gene P

37. CO2 Storage Benchmark Problems A Benchmark-Study on Problems Related to CO2 Storage in Geological formations, Summary and Discussion of the Results H. Class, A. Ebigbo, R. Helming et al., 2008

38. Benchmark Problem 1.1CO2 Plume Evolution and Leakage via Abandoned Well Objective Quantification of leakage rate in deep aquifer @2840-3000 m Output 1- Leakage rate = %CO2 mass flux/injection rate 2- Max. leakage value 3- Leakage value at 1000 d K = 20 md f=0.15 P = 3.08x104 KPa

39. Benchmark Problem 1.1Leakage Rate of CO2 CO2 BT: 10 days Peak Leakage value: 0.23% Final leakage value: 0.11% Agrees with semi-analytic solution (Nordbotten et al.)

40. Comparison with Published Resultsat 80 days Ebigbo et al., 2007 IPARS-COMP Gas Saturation Pressure

41. Frio Brine CO2 Injection Pilot Bureau of Economic Geology Jackson School Of Geosciences The University of Texas at Austin Funded by DOE NETL

42. Frio Brine Pilot Site • Injection interval: 24-m-thick, mineralogically complex fluvial sandstone, porosity 24%, Permeability 2.5 D • Unusually homogeneous • Steeply dipping 16 degrees • 7m perforated zone • Seals  numerous thick shales, small fault block • Depth 1,500 m • Brine-rock, no hydrocarbons • 150 bar, 53 C, supercritical CO2 Injection interval Oil production From Ian Duncan

43. Frio Modeling Effort Stair stepped approximation on a 50x100x100 grid (~70,000 active elements) has been generated from the given data. Figure shows porosity in the given and approximated data.

44. Solution profiles Pressure and close-up of top-view of gas (CO2) saturation at t=33 days. Simulations on bevo2 cluster at CSM, ICES on 24 processors.

45. CO2 Plume Transport CO2 saturation as seen below the shale barrier at t=2 and 33 days. Breakthrough time is observed to be close to 2 days.

46. Current Research Activities at CSM Model CO2 injection either in deep saline aquifers or depleted oil and gas reservoirs using compositional and parallel reservoir simulator (IPARS-CO2) • Large scale parallel computing • Efficiency with different solvers • Couple IPARS-CO2 with geochemistry • Couple IPARS with geomechanics • Enhance EOS model and physical property models (effect of salt, hysteresis, etc) • Data sources, field sites, practical applications (in collaboration with Duncan from BEG at UT) • Gridding and a posteriori error estimators • Optimization • Risk and uncertainty analysis