3.1 Introduction and Terminology

Machine Learning (ML) is a field of computer science that uses statistical techniques to give computer systems the ability to "learn" (i.e. progressively improve performance on a specific task) with data, without being explicitly programmed. ML is subfield of Artificial Intelligence (AI) which is defined as a branch of computer science dealing with the simulation of intelligent behavior in computers.

The Machine learning plugin links the OpendTect Pro environment to the Keras (TensorFlow) and Scikit Learn platforms. With the Machine learning plugin, OpendTect users can do Clustering, Classification and Regression of the following objects:

  1. Wells only e.g. for predicting missing logs, markers, stratigraphic trends, litho-facies, clustering logs etc.

  2. Seismic only e.g. for clustering 3D seismic, imaging faults, chimneys, salt bodies, to predict missing traces, picking velocities, picking mutes, increasing spatial and temporal sampling rates, transforming seismic data to broadband etc.

  3. Seismic & wells e.g. for predicting reservoir properties, explaining patterns, tying wells, etc.

  4. Neural Network e.g. for predicting pattern and properties using the conventional Neural Network.

 

The Machine Learning plugin can be accessed as pointed in the image below:

 

The Machine Learning Control Center is developed with the same look-and-feel as other control center plugins of OpendTect.

First we select the type of data we are working with: Wells, Seismic, Seismic + Wells, or Neural Networks. The latter refers to the pre-existing Neural Networks plugin that is now started from inside the ML control center.

Next we select the workflow we want the execute from the available options (see images below). The UI then guides us through the selected workflow.

Typically, each workflow consists of three steps, which are captured in three tabs: Select input data -> Train Model -> Apply Trained Model.

 

ML workflows for: Wells, Seismic, Seismic+Wells and the predecessor Neural Networks plugin:

 

 

Terminology

In this Chapter we will use the following terms:

  1. Accuracy and Overfitting
    1. A Model is accurate if it can be said to predict as well in the validation data as it did in the training data.
    2. An overly complex Model may be trained such that it is very accurate for predictions on the Training Set but does not generalize well.
    3. Overfitting results when the Model no longer describes the relationship between Features and Target, but instead is trained to output only the Training data.
  2. Algorithm
    1. A process, set of rules or mathematical relationships used to calculate a solution (or solve) a problem.
    2. Typical Machine Learning Algorithms are Regression, Random Forest, Boosting, Neural Networks, Self-Organizing Maps (SOMs).
  3. Classification
    1. A prediction method that assigns a data point (with some feature coordinates) to a predefined category or bin.
    2. Classification is strictly speaking a Supervised learning approach (target labels are assigned by experts).
    3. Classification is nowadays also used in the context of Unsupervised learning approaches (target labels are generated by the model).
    4. In this document we use Classification for both Supervised and Unsupervised learning approaches.
  4. Clustering
    1. An Unsupervised learning approach aimed at Organizing data in groups with similar structure.
  5. Convolutional Neural Network (CNN)
    1. A specific type of neural network in which a (1D, 2D, 3D) image is convolved with a set of filters and the filtered output is passed on to the next layer in the network.
    2. The filter coefficients are found in a supervised learning run.
    3. The LeNet and Unet models implemented in OpendTect Machine Learning plugin are CNNs.
    4. Webinar: Machine Learning Part 2.
  6. Feature
    1. An independent variable such an observable quantity that has been previously recorded.
    2. Features can also be engineered by combining multiple features or applying transformations.
    3. Features are the inputs of Models.
  7. LeNet Model
    1. Convolutional Neural Network introduced by Yann LeCunn
    2. Used in OpendTect for Seismic Bodies detection workflows (e.g. creating a Chimney Cube with a deep learning model instead of with a shallow MLP type of neural network).
    3. Webinar: Machine Learning Part 2; Publications: http://yann.lecun.com/exdb/lenet/
  8. Model
    1. A mathematical representation of some real-world process.
    2. A predictive model attempts to produce an outcome based on past experience.
  9. Multi-Layer-Perceptron (MLP)
    1. A fully-connected neural network with an input layer, one or more hidden layers and an output layer.
    2. The non-linear transformation happens in the hidden layer(s) when the sum of the weights multiplied by the input features passes through a sigmoidal shape function.
    3. MLP networks are supervised networks for classification and regression problems.
    4. MLPs are nowadays referred to as “shallow” networks as a distinction between conventional neural networks and newer deep learning models.
    5. In OpendTect MLPs are implemented in the pre-existing Neural Networks plugin and in various workflows with Scikit Learn models.
    6. Webinar: Machine Learning Part 1.
  10. Segmentation
    1. Strictly speaking Segmentation is the same as Clustering, i.e. an Unsupervised learning approach.
    2. Segmentation is nowadays also used for Supervised learning methods.
    3. We primarily use Segmentation in the context of image-to-image predictions whereby the Target is an image with labeled information (aka mask).
  11. Target
    1. Defined in Statistics as the dependent variable.
    2. It is the output (or prediction) of the Model or variable.
  12. Test Set
    1. A data set that is (automatically) split off by random selection from the Training Set. The Test Set is used during training to check for signs of Overfitting.
    2. Examples from the Test Set are passed through the Model to compute the error. In contrast to examples from the training set the errors of the Test Set are not used to update the weight function (gradients) during training.
    3. Training should ideally be stopped when the error on the Test Set has reached its minimum. Prolonged training may further decrease the error on the Training Set while the error on the test set increases (overfitting).
    4. The Test Set differs from the Validation set in that the Validation set is not used during training whereas the Test Set is.
    5. TensorBoard does not make a distinction between Test Set and Validation.
  13. Training (“Learning”)
    1. The process of creating a Model from the Training data using some algorithm.
    2. The data is fed into the algorithm, which learns a representation of the relationship between Feature(s) and Target(s).
    3. Supervised Learning is an approach to learning under supervision from an intelligent source. For example, learn by advice, or learn by example.
    4. Unsupervised Learning is a competitive learning approach aimed at finding structure in the data, thereby extracting relevant properties or features.
  14. Training Set
    1. A (cleaned up, pre-processed) data set that is used by the learning algorithm to find potentially predictive relationships.
    2. Typically, the Training data set is a subset of the entire dataset, with some portion being held aside for Validation.
    3. For seismic applications a Training Set can also be synthesized. This is especially useful in cases where we do not have sufficient reliable examples to train on. In OpendTect we synthesize data (1D, 2D and 3D) in SynthRock. Also, the Unet Fault predictor was trained on synthetic data.
    4. The key to successful machine learning applications is creating a training set that is truly representative of the problem we try to solve.
  15. Unet Model
    1. An auto-encoder - decoder type of Convolutional Neural Network used in Segmentation and Regression problems.
    2. Webinar: Machine Learning Part 2; Publications: https://link.springer.com/chapter/10.1007/978-3-319-24574-4_28
  16. Unsupervised Vector Quantizer (UVQ)
    1. Network for segmenting (clustering) data into a user-defined number of segments (UVQ is similar to K-means clustering).
    2. Cluster centers are found by unsupervised learning; Random prototype vectors are updated until they fall in cluster centers; Post-training cluster centers are sorted to ensure that similar clusters are displayed with neighboring colors in the segmentation map (UVQ + sorting approximates a Kohonen map, which sorts during training as it updates not only the winner but also the nearest neighbor).
    3. In the application phase the trained UVQ network compares input vectors to ‘cluster center’ vectors; Output is the index of ‘winning’ cluster and ‘match’ (confidence measure between 1 and 0).
    4. UVQ is used in seismic facies interpretation workflows. Typically, it is used to cluster seismic waveforms around mapped horizons but it can also be used to cluster 3D bodies.
    5. Quick UVQ is a workflow for waveform segmentation, i.e. clustering trace segments extracted around a mapped horizon.
    6. In OpendTect UVQ is implemented in the pre-existing Neural Networks plugin.
    7. Webinar: Machine Learning Part 1.
  17. Validation Set
    1. A data set separate from the Training Set, yet having the same predictive features and targets as the training data set, e.g. a well with existing logs that is set aside as blind test well. Please note that TensorBoard does not make a distinction between Test Set (see above) and Validation Set.
    2. Comparison of the predictions versus known results in the training and validation data speak to the accuracy and generality of the model.

 

 

You can find the complete dGB Machine Learning Webinar series here:

 

Other dGB resources include:

 

And a selection of relevant dGB Machine Learning workflows (in .pdf):

 

For a general (non dGB) beginner's tutorial course in TensorFlow 2.0, please watch this YouTube video.