Petroleum Reservoir Management Based on Approximate Dynamic Programming

1. Petroleum Reservoir Management Based on Approximate Dynamic Programming Zheng Wen, Benjamin Van Roy, Louis Durlofsky and Khalid Aziz Smart Field Consortium, Stanford University Basis Function Selection Motivation Case 1: General Primary Production Case 3: Comparison with Gradient-Based Method • Shortage of energy resources calls for better petroleum reservoir management policies • Optimizing decision-making in reservoir management is challenging • Large-scale nonlineardynamic optimization problem • Complicated constraints on control • Current optimization techniques have various limitations • We propose optimization algorithms based on Approximate Dynamic Programming (ADP) to solve this problem • Basis functions include: • Global variables: constant, total oil, total water, oil/water ratio • Basis functions constructed by Proper Orthogonal Decomposition (POD): • Linear dynamics with complex cost function • Penalize on low BHP • Geological model is a modified portion of SPE 10 • 35×35×1blocks,4production wells • We use fixed basis functions in this case • Baseline is chosen as the best result of 500 constant-control simulations and 500 randomized-control simulations • We apply both ADP(with fixed basis functions) and Gradient-Based Algorithm to an example and compare their performances: • Black-oil Model • 40×40×1 blocks, 4 production wells and 4 injection wells • Constraints on control: Bounded BHPs • We rerun gradient-based method for 200 different starting points, and record the best and the worst local optimum Reservoir Management as an Optimization Problem • Petroleum Reservoir can be modeled as a nonlinear dynamic system • State x: Pressure or Saturation • Control u: Bottom-Hole Pressure or Well Flow Rate • L(x,u): Instantaneous cost • System dynamics are determined by • Fluid dynamics • Mass balance equation • Geological model • Objective: System (Petroleum Reservoir) Instantaneous cost L(x,u) Compute Weights: Smoothing Reduced Linear Programming (RLP) with Regularization • An implementable approximation of LP approach in DP Control State x Control Policy Case 2: Black Oil Model with Simple Constraints Case 4: Black-Oil Model with Complicated Constraints L1Regularization Sampled States • Classical model in Petroleum Engineering • Two phase, two component model (oil, water) • Produce oil (with some water) by injecting water • The dynamic model is highly nonlinear • Geological Model is also a modified portion of SPE-10 • 35×40×1 blocks, 4 production wells and 1 injection well • Constraints on controls: BHPs are bounded • We use fixed basis functions in this case • Improvement is • 15% compared with myopic control • 8% compared with myopic-then-stop control • water saturation is more symmetric after optimization • Black-oil Model • 40×40×1 blocks, 2 production wells and 2 injection wells • Constraints on controls: • Bounded BHP • Maximum water cut of each production well • Maximum liquid injection/production rates • Minimum oil production rate • Apply ADP with adaptive basis functions • Baseline: best result of 100 feasible control strategies • Result: 19% improvement over baseline (see Figure above) Slack Variable (Smoothing) Weights ADP Algorithm (Fixed Basis Function) Sample States Based on Randomized Baseline Strategy Input for SVD Constraints Basis Fun Brief Overview of ADP Construct Basis Functions by POD Compute Weights • Dynamic Programming (DP) potentially achieves global optimum but suffers from the “curse of dimensionality” • ADP tries to keep DP’s merits but overcome the dimensionality curse • Approach: approximate the cost-to-go function as a linear regression of a set of basis functions: • Question: how to choose basis functions and weights? • Many ADP algorithms could be used to compute weights for given basis functions; • Choice of basis functions is highly problem dependent • For reservoir production problem, basis functions should contain “enough information” about future total cost Basis Fun Weights Conclusion and Future Work Approximate Cost-to-go Function • We have applied ADP to reservoir optimization problems and compared its performance with baseline/other optimization algorithms • ADP has been shown to be a promising approach for reservoir production problems • Tentative future works include: • Test ADP extensively for realistic 3D examples • Compare with other optimization techniques • Model uncertainty of geological parameters Solve Sub-Problem Control Strategy ADP Algorithm (Adaptive Basis Function) Sample States Based on Randomized Baseline Strategy Input for SVD Constraints Basis Fun Basis Function Selection Construct Basis Functions by POD Compute Weights Basis Fun Weights Approximate Cost-to-go Function Strategy Evaluation Control Strategy