`dgbpy.torch_classes`

Module Contents

Classes

`OnnxTorchModel`
`Net`
`dGBLeNet`
`AdaptiveLR`
`Callback`
`TrainEvalCallback`
`AvgStats`
`AvgStatsCallback`
`ProgressBarCallback`
`BokehProgressCallback`	Send progress message to bokeh
`EarlyStoppingCallback`
`TensorBoardLogCallback`
`LogNrOfSamplesCallback`
`TransformCallback`
`LRSchedulerCallback`
`Trainer`
`ResidualBlock`	Residual Block within a ResNet CNN model
`Concatenate`
`DownBlock`	A helper Module that performs 2 Convolutions and 1 MaxPool.
`UpBlock`	A helper Module that performs 2 Convolutions and 1 UpConvolution/Upsample.
`dGBUNet`
`UNet`	activation: 'relu', 'leaky', 'elu'
`UNet_VGG19`
`TrainDatasetClass`
`TestDatasetClass`
`DatasetApply`
`DataPredType`	Generic enumeration.
`OutputType`	Generic enumeration.
`DimType`	Generic enumeration.
`TorchUserModel`	Abstract base class for user defined Torch machine learning models

Functions

`Tensor2Numpy`(tensor)
`Numpy2tensor`(nparray)
`hasFastprogress`()
`ignore_index`(out, target, ignore_index=-1)
`flatten`(out, target)
`jaccard`(out, target)
`accuracy`(out, target)
`f1`(out, target)
`mae`(out, target)
`reformat_str`(name)
`create_resnet_block`(input_filters, output_filters, num_residuals, ndims, first_block=False)
`autocrop`(encoder_layer: torch.Tensor, decoder_layer: torch.Tensor)	Center-crops the encoder_layer to the size of the decoder_layer,
`conv_layer`(dim: int)
`get_conv_layer`(in_channels: int, out_channels: int, kernel_size: int = 3, stride: int = 1, padding: int = 1, bias: bool = True, dim: int = 2)
`conv_transpose_layer`(dim: int)
`get_up_layer`(in_channels: int, out_channels: int, kernel_size: int = 2, stride: int = 2, dim: int = 3, up_mode: str = 'transposed')
`maxpool_layer`(dim: int)
`get_maxpool_layer`(kernel_size: int = 2, stride: int = 2, padding: int = 0, dim: int = 2)
`get_activation`(activation: str)
`get_normalization`(normalization: str, num_channels: int, dim: int)

Attributes

mlmodels

dgbpy.torch_classes.Tensor2Numpy(tensor)

dgbpy.torch_classes.Numpy2tensor(nparray)

dgbpy.torch_classes.hasFastprogress()

class dgbpy.torch_classes.OnnxTorchModel(filepath: str)

Bases: dgbpy.dgbonnx.OnnxModel

__call__(self, inputs)

convert_to_torch(self)

eval(self)

class dgbpy.torch_classes.Net(model_shape, output_classes, dim, nrattribs)

Bases: torch.nn.Module

after_cnn(self, x)

forward(self, x)

class dgbpy.torch_classes.dGBLeNet(model_shape, output_classes, dim, nrattribs, predtype)

Bases: torch.nn.Module

after_cnn(self, x)

forward(self, x)

dgbpy.torch_classes.ignore_index(out, target, ignore_index=-1)

dgbpy.torch_classes.flatten(out, target)

dgbpy.torch_classes.jaccard(out, target)

dgbpy.torch_classes.accuracy(out, target)

dgbpy.torch_classes.f1(out, target)

dgbpy.torch_classes.mae(out, target)

dgbpy.torch_classes.reformat_str(name)

class dgbpy.torch_classes.AdaptiveLR(optimizer, initial_lrate, epochdrop)

Bases: torch.optim.lr_scheduler._LRScheduler

get_lr(self)

class dgbpy.torch_classes.Callback

_order = 0

set_runner(self, run)

__getattr__(self, k)

property name(self)

__call__(self, cb_name)

class dgbpy.torch_classes.TrainEvalCallback

Bases: Callback

_order = 1

begin_fit(self)

begin_batch(self)

begin_epoch(self)

begin_validate(self)

class dgbpy.torch_classes.AvgStats(metrics)

reset(self)

property all_stats(self)

property avg_stats(self)

accumulate(self, run)

class dgbpy.torch_classes.AvgStatsCallback(metrics)

Bases: Callback

_order = 2

begin_fit(self)

begin_epoch(self)

after_loss(self)

after_epoch(self)

class dgbpy.torch_classes.ProgressBarCallback

Bases: Callback

_order

begin_fit(self)

after_fit(self)

after_batch(self)

begin_epoch(self)

begin_validate(self)

set_pb(self)

class dgbpy.torch_classes.BokehProgressCallback

Bases: Callback

Send progress message to bokeh

_order

begin_batch(self)

begin_epoch(self)

after_batch(self)

after_epoch(self)

after_fit(self)

before_fit_chunk(self)

begin_fold(self)

class dgbpy.torch_classes.EarlyStoppingCallback(patience)

Bases: Callback

_order = 3

le_gr(self): Return the right ( > ) or ( < ) operator.

begin_fit(self)

after_epoch(self)

after_fit(self)

class dgbpy.torch_classes.TensorBoardLogCallback

Bases: Callback

_order = 10

begin_batch(self)

after_epoch(self)

class dgbpy.torch_classes.LogNrOfSamplesCallback

Bases: Callback

_order = 0

begin_fit(self)

begin_fold(self)

class dgbpy.torch_classes.TransformCallback

Bases: Callback

begin_epoch(self)

class dgbpy.torch_classes.LRSchedulerCallback(scheduler)

Bases: Callback

after_epoch(self)

exception dgbpy.torch_classes.CancelTrainException

Bases: Exception

Common base class for all non-exit exceptions.

exception dgbpy.torch_classes.CancelEpochException

Bases: Exception

Common base class for all non-exit exceptions.

exception dgbpy.torch_classes.CancelBatchException

Bases: Exception

Common base class for all non-exit exceptions.

class dgbpy.torch_classes.Trainer(model: torch.nn.Module, criterion: torch.nn.Module, optimizer: torch.optim.Optimizer, scheduler, device: torch.device, training_DataLoader: torch.utils.data.Dataset, validation_DataLoader: torch.utils.data.Dataset = None, metrics=None, tensorboard=None, epochs: int = 100, earlystopping: int = 5, imgdp=None, cbs=None, silent=None, tofp16=False)

ALL_CBS

init_callbacks(self, cbs)

set_metrics(self, custom_metrics)

add_cbs(self, cbs)

remove_cbs(self, cbs)

compute_loss_func(self)

one_batch(self, i, input, target)

all_batches(self)

do_begin_epoch(self, epoch)

fit_one_chunk(self, ichunk, cbs)

train_fn(self)

fit(self, cbs=None)

__call__(self, cb_name)

class dgbpy.torch_classes.ResidualBlock(input_channels, num_channels, use_1x1_conv=False, strides=1, ndims=3)

Bases: torch.nn.Module

Residual Block within a ResNet CNN model

forward(self, X)

shape_computation(self, X)

initialize_weights(self)

dgbpy.torch_classes.create_resnet_block(input_filters, output_filters, num_residuals, ndims, first_block=False)

dgbpy.torch_classes.autocrop(encoder_layer: torch.Tensor, decoder_layer: torch.Tensor): Center-crops the encoder_layer to the size of the decoder_layer, so that merging (concatenation) between levels/blocks is possible. This is only necessary for input sizes != 2**n for ‘same’ padding and always required for ‘valid’ padding.

dgbpy.torch_classes.conv_layer(dim: int)

dgbpy.torch_classes.get_conv_layer(in_channels: int, out_channels: int, kernel_size: int = 3, stride: int = 1, padding: int = 1, bias: bool = True, dim: int = 2)

dgbpy.torch_classes.conv_transpose_layer(dim: int)

dgbpy.torch_classes.get_up_layer(in_channels: int, out_channels: int, kernel_size: int = 2, stride: int = 2, dim: int = 3, up_mode: str = 'transposed')

dgbpy.torch_classes.maxpool_layer(dim: int)

dgbpy.torch_classes.get_maxpool_layer(kernel_size: int = 2, stride: int = 2, padding: int = 0, dim: int = 2)

dgbpy.torch_classes.get_activation(activation: str)

dgbpy.torch_classes.get_normalization(normalization: str, num_channels: int, dim: int)

class dgbpy.torch_classes.Concatenate

Bases: torch.nn.Module

forward(self, layer_1, layer_2)

class dgbpy.torch_classes.DownBlock(in_channels: int, out_channels: int, pooling: bool = True, activation: str = 'relu', normalization: str = None, dim: str = 2, conv_mode: str = 'same')

Bases: torch.nn.Module

A helper Module that performs 2 Convolutions and 1 MaxPool. An activation follows each convolution. A normalization layer follows each convolution.

forward(self, x)

class dgbpy.torch_classes.UpBlock(in_channels: int, out_channels: int, activation: str = 'relu', normalization: str = None, dim: int = 3, conv_mode: str = 'same', up_mode: str = 'transposed')

Bases: torch.nn.Module

A helper Module that performs 2 Convolutions and 1 UpConvolution/Upsample. An activation follows each convolution. A normalization layer follows each convolution.

forward(self, encoder_layer, decoder_layer): Forward pass Arguments:

encoder_layer: Tensor from the encoder pathway decoder_layer: Tensor from the decoder pathway (to be up’d)

class dgbpy.torch_classes.dGBUNet(in_channels, model_shape, out_channels, predtype)

Bases: torch.nn.Module

forward(self, x)

class dgbpy.torch_classes.UNet(in_channels: int = 1, out_channels: int = 2, n_blocks: int = 1, start_filters: int = 32, activation: str = 'relu', normalization: str = 'batch', conv_mode: str = 'same', dim: int = 2, up_mode: str = 'transposed')

Bases: torch.nn.Module

activation: ‘relu’, ‘leaky’, ‘elu’ normalization: ‘batch’, ‘instance’, ‘group{group_size}’ conv_mode: ‘same’, ‘valid’ dim: 2, 3 up_mode: ‘transposed’, ‘nearest’, ‘linear’, ‘bilinear’, ‘bicubic’, ‘trilinear’

static weight_init(module, method, **kwargs)

static bias_init(module, method, **kwargs)

initialize_parameters(self, method_weights=nn.init.xavier_uniform_, method_bias=nn.init.zeros_, kwargs_weights={}, kwargs_bias={})

forward(self, x: torch.tensor)

__repr__(self)

class dgbpy.torch_classes.UNet_VGG19(model_shape, out_channels, nrattribs)

Bases: torch.nn.Module

forward(self, x)

class dgbpy.torch_classes.TrainDatasetClass(imgdp, scale, transform=list(), transform_copy=False)

Bases: torch.utils.data.Dataset

__len__(self)

__getitem__(self, idx)

Details:

idxIndex of the sample to be retrieved.

using the expected multiplied number of samples by the trfm_multiplier

set_chunk(self, ichunk): Set the chunk to be used for training

set_fold(self, ichunk, ifold): Set the fold for a particular chunk to be used for training

set_transform_seed(self): Set different seed for each epoch

get_data(self, trainchunk, ichunk): Get the data from the chunk

set_transforms(self, info): Set the transforms to be applied to the data

_adaptShape(self, X, Y): Adapt the shape of the data to the expected shape of the network

class dgbpy.torch_classes.TestDatasetClass(imgdp, scale)

Bases: torch.utils.data.Dataset

__len__(self)

transformer(self, image, label, index)

set_chunk(self, ichunk)

set_fold(self, ichunk, ifold)

get_data(self, validchunk, ichunk)

__getitem__(self, index)

class dgbpy.torch_classes.DatasetApply(X, info, isclassification, im_ch, ndims)

Bases: torch.utils.data.Dataset

__len__(self)

__getitem__(self, index)

class dgbpy.torch_classes.DataPredType

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

Continuous = Continuous Data

Classification = Classification Data

Segmentation = Segmentation

Any = Any

class dgbpy.torch_classes.OutputType

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

Pixel = 1

Image = 2

Any = 3

class dgbpy.torch_classes.DimType

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

D1 = 1

D2 = 2

D3 = 3

Any = 4

classmethod is_valid(cls, dim_type)

class dgbpy.torch_classes.TorchUserModel

Bases: abc.ABC

Abstract base class for user defined Torch machine learning models

This module provides support for users to add their own machine learning models to OpendTect.

It defines an abstract base class. Users derive there own model classes from this base class and implement the _make_model static method to define the structure of the torch model. The users model definition should be saved in a file name with “mlmodel_” as a prefix and be at the top level of the module search path so it can be discovered.

The “mlmodel_” class should also define some class variables describing the class: uiname : str - this is the name that will appear in the user interface uidescription : str - this is a short description which may be displayed to help the user predtype : DataPredType enum - type of prediction (must be member of DataPredType enum) outtype: OutputType enum - output shape type (OutputType.Pixel or OutputType.Image) dimtype : DimType enum - the input dimensions supported by model (must be member of DimType enum)

mlmodels = []

static findModels()

Static method that searches the PYTHONPATH for modules containing user defined torch machine learning models (TorchUserModels).

The module name must be prefixed by “mlmodel_”. All subclasses of the TorchUserModel base class is each found module will be added to the mlmodels class variable.

static findName(modname): Static method that searches the found TorchUserModel’s for a match with the uiname class variable

Parameters

modname : str Name (i.e. uiname) of the TorchUserModel to search for.

Returns

an instance of the class with the first matching name in the mlmodels list or None if no match is found

static getModelsByType(pred_type, out_type, dim_type): Static method that returns a list of the TorchUserModels filtered by the given prediction, output and dimension types

Parameters

pred_type: DataPredType enum The prediction type of the model to filter by out_type: OutputType enum The output shape type of the model to filter by dim_type: DimType enum The dimensions that the model must support

Returns

a list of matching model or None if no match found

static getNamesByType(pred_type, out_type, dim_type)

static isPredType(modelnm, pred_type)

static isOutType(modelnm, out_type)

static isClassifier(modelnm)

static isRegressor(modelnm)

static isImg2Img(modelnm)

abstract _make_model(self, model_shape, nroutputs, nrattribs)

Abstract static method that defines a machine learning model.

Must be implemented in the user’s derived class

Parameters

input_shape : tuple nroutputs : int (number of discrete classes for a classification) Number of outputs learnrate : float

Returns

a compiled torch model

model(self, model_shape, nroutputs, nrattribs): Creates/returns a compiled torch model instance

Parameters

nroutputs : int (number of discrete classes for a classification) Number of outputs

Returns

a pytorch model architecture

dgbpy.torch_classes.mlmodels

dgbpy.torch_classes

Module Contents

Classes

Functions

Attributes

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

`dgbpy.torch_classes`