White Space Energy | BY Norbert Dolle
Introduction
Well trajectory planning is a high-stake and complex multi-disciplinary work activity for Oil & Gas operators. The work involves experts from geoscience, reservoir engineering, drilling, completions and facilities. Each are using specialist software to evaluate and define reservoir targets, subsurface hazards and engineering constraints. “Likes” and “dislikes” of trajectory options are expressed in different terms by the various disciplines. This often leads to an iterative and time-consuming process, influenced by human bias. Time quickly becomes a limiting factor, with a business risk of unrealized value due to incomplete understanding of the full option space and associated uncertainties, risks and rewards.
To mitigate the above challenges, we have developed a collaborative game-based approach to well trajectory planning supported by Artificial Intelligence (AI). This approach has been tested using Equinor’s open source Volve dataset, which demonstrates the potential to significantly reduce cycle time and improve decision quality.
Our Game Approach
Oil & Gas companies generate value through the quality of their decisions. Often, these decisions are very complex. There are millions of choices, conflicting business drivers, lots of uncertainty, hidden bias, and so forth. These complexities all add up, which makes good decision making very hard – for humans.
In contrast to humans, Game AI is designed to handle such complexities. These latest AI algorithms mimic human ability to learn from experience and have become famous for achieving super-human performance in sophisticated and complex games such as Chess, Go, StarCraft and Poker.
In this paper we describe our game approach to well trajectory planning, and demonstrate how AI can be used to support humans in making faster and better decisions.
The approach will be described using the following three steps:
- Building the game
- Playing the game
- Analysing strategies
Step 1 – Building the game
When seen through a gamification lens, well trajectory planning is very similar to a video game such as PacMan. Players can “collect” rewards in the game, such as remaining oil or gas. There are also items to avoid, such as drilling hazards. And there are movement rules, things like maximum dog leg severity, total well length and trajectory tortuosity. In the next sections we will describe how these rewards, obstacles and movement rules are built into a case-specific game environment.
Rewards, obstacles and movement rules are available as existing static, dynamic and engineering models, interpretations and opinions. The key differentiator of the game approach (in contrast to the current siloed ways of working) is that contributions from all disciplines are incorporated in a single (game) environment, whichsignificantly improves integration and strengthens collaboration.
For this project we used Equinor’s open-source Volve dataset as basis. To make the case more interesting as a demonstrator of our Game AI approach, we have assumed that a secondary gas cap has been created in the first phases of the field development. The remaining oil is consequently a thin oil rim of about 30 meters thickness. Furthermore, we have assumed that gas cap and aquifer strenghts are such that the highest recovery efficiency is associated with offtake from the middle of the rim, and therefore the ideal placement of (horizontal) wells is exactly in between the Gas Oil Contact (GOC) and the Oil Water Contact (OWC). The resulting positive reward system, i.e. the “points” that can be scored in the game, is schematically shown in Figure 1 as a multiplication of net pore volume, mobile oil saturation and recovery efficiency.
For the purpose of the actual demonstrator, this reward system was further expanded with recoverable oil per length of reservoir penetration (barrels per m) and a proxy value per barrel ($/bbl) to reflect economics.
While the oil represents the positive reward in this well trajectory planning game, there are also negative rewards or penalties that can be accumulated. These “obstacles” are shown in Figure 2 and consist of:
- Existing wells. Existing well include a minimum separation distance for anti-collision purposes. Hitting one of these leads to a negative reward of minus infinity, therefore means “game over”.
- Faults. The negative reward is angle-dependent, reflecting that the drilling risks vary by the angle under which the fault plane is penetrated.
- Shallow gas. Drilling through shallow gas can be a significant drilling hazard, in particular when being unprepared for it. The severity of the negative reward can be varied based on the perceived managability of the risk, and to evaluate robustness of the ultimate highest ranking well trajectories.
Drilling and completion cost can equally be included in the reward system, or added as post-processing in the analysis (Step 3).
The movement rules in this game are defined by parameters such as surface location, kick-off depth and maximum dog leg constraints. Other design considerations, such as maximum inclinations and/or allowable azimuths for casing design, can either be brought in as rules or taken into account in the analysis of the strategies.
All the above reward system elements and movement rules are programmed into a custom game environment using Python. This makes the approach extremely flexible, which is one of its key strengths. For example, the game environment in this project is entirely deterministic – but stochasticity (e.g. representing subsurface uncertainty) can be easily built in when that is preferred for decision making.
Step 2 – Playing the game
The game is played by a virtual player, our AI assistant. The aim of this particular demonstrator game was to explore the option space of drillable wells and find the “preferred next well to drill”.
Since we want to give decision makers an understanding of the full option space of valuable and feasible wells, the AI assistant has a simple task: design drillable wells that collect as much reward as possible whilst avoiding or minimising penalties.
Designing drillable well trajectories involves a sequence of decisions, which makes the process very suitable for AI algorithms. Different solver architectures, or algorithms, can be used to play this game. This is similar to how companies such as Google-owned DeepMind develop customized solvers for games such as Go and StarCraft. Each architecture has pros and cons and very often it is the problem definition (i.e. the game objectives) that determines which solver is most effective. For this project, three solver architectures were considered: Reinforcement Learning, Evolutionary Algorithms and Tree Search algorithms (see Figure 3).
We landed on a Tree Search algorithm with an evolutionary layer on top as the most fit-for-purpose architecture in this case, mainly owing to its performance (i.e. speed) and exploration capability (i.e. it looks “wide” in the option space).
The strength and flexiblity of our approach came through in this project by using the same game set up to design, evaluate and compare trajectories for different well types: from new wells at different surface locations, to shallow and deep sidetracks. For the AI, these are simply different starting points and movement rules (e.g. reflecting the range of feasible whipstock settings for sidetracks).
Step 3 – Analysing strategies
It is one thing to have an AI that designs thousands of feasible well trajectories per minute. It is another thing to make this data available for effective analysis and quality decision making. Decision making in Oil & Gas is often a complex task due to competing business drivers (safety, sustainability, cost, value). In the case of well trajectory planning, the preferred choice can depend on many things. From quantifiable parameters such as oil price, cost, budget availability, drilling contractor performance and subsurface uncertainty, to softer items such as risk appetite, experience and gutfeel. The analysis of the game play results should therefore allow all these aspects to be taken into account.
In general terms we use a 2-step approach to facilitate the analysis and decision-making:
1. Reduce the number of options without risking decision quality. This can be done in various ways. We have previously used clustering techiques for this purpose, i.e. lumping sets of “similar” well trajectories into a single trajectory that is representive for the entire set. Another option, used in the project, is to only consider the trajectories that are on (or near to) the Pareto front (Ref. Jahan et al).
2. Present the remaining options in a bespoke analysis dashboard. The dashboard visualises the options, allows interactive filtering and quantifies the trade-offs between key decision metrics. The dashboard used for the project described in this paper is presented in Figure 4.
The collaborative aspects of the game come to fruition when the integrated team uses the dashboard to discuss and evaluate options. The interactive overview of considerations and opinions stimulates a healthy cross-disciplinary challenge and helps the team to understand the trade-offs and/or to ask “what if” questions to further improve robustness of the preferred choice(s).
Key Take Aways
This article describes how we have developed a game-based approach to well trajectory design supported by Artificial Intelligence (AI). This approach has been tested using the Volve dataset, which demonstrates the potential to significantly reduce cycle time and improve decision quality.
Based on the Volve dataset we have built a game environment that incorporates subsurface rewards, obstacles and movement rules. Using AI as a virtual player in this environment, our approach leads to a complete overview of the option space, and associated trade-offs across different business drivers, at a fraction of the time that it takes manual workflows to give similar insight. The results from the AI assisted game play are presented in a bespoke analysis dashboard to facilitate decision making.
This game-based approach to Well Trajectory Planning offers a significant opportunity for better and faster decisions in situations where manual workflows are overly time consuming, potentially biased or both. This includes fields with complex geology (e.g. stacked and compartmentalised with multiple drilling hazards), mature assets with congested surface and subsurface infrastructure (where picking well trajectories could be a significant issue due to anti-collision constraints) and field developments where the trade-offs between drilling and surface expenditures pose a challenging optimisation problem.
Acknowledgements
We thank Equinor for supporting this joint effort and the valuable discussions along the way, as well as for making the Volve dataset publicly available
Note: this article is co-authored by Einar Landre & Olav Ragnvald Hansen from Equinor, and Tom Savels & Germonda Reijnen from White Space Energy