Speaker
Description
Reinforcement Learning (RL) has recently gained traction as a promising tool for solving challenging control and optimization problems in porous media. In particular, subsurface reservoir management is a critical application domain, where optimal injection and production strategies can significantly enhance recovery while minimizing operational costs. However, purely model-free RL approaches often demand excessive computational budgets due to the large number of high-fidelity simulations needed during training.
In this work, we investigate model-based reinforcement learning (MBRL) as a more sample-efficient alternative for controlling two-phase flow in heterogeneous reservoirs. We focus on two state-of-the-art MBRL algorithms—Probabilistic Ensembles with Trajectory Sampling (PETS) and Model-Based Policy Optimization (MBPO)—which learn surrogate models of reservoir dynamics to either plan control actions (PETS) or generate synthetic rollouts for policy training (MBPO). We implement and couple these methods with the open-source simulator JutulDarcy, enabling flexible experimentation on waterflooding scenarios with multiple injectors and producers.
Our numerical experiments highlight distinct advantages and trade-offs of each MBRL approach. PETS achieves high data efficiency by iteratively refining action sequences through an uncertainty-aware planning routine but can occasionally settle in local optima under complex geology. In contrast, MBPO demonstrates robust performance in navigating high-dimensional control spaces, although it is more sensitive to hyperparameters and requires careful tuning to avoid training instabilities. Through ablation studies, we examine how key factors—such as planning horizon, surrogate model supervision length, and policy update frequency—affect convergence and policy quality in reservoir management tasks.
Overall, our results illustrate that MBRL provides a powerful framework for constructing computationally tractable proxy models and efficient control policies for porous media flow problems. These findings open new directions for RL applications in reservoir engineering, promising improved scalability and adaptability compared to classical optimization workflows.
| Country | UK |
|---|---|
| Acceptance of the Terms & Conditions | Click here to agree |
