dgbpy.mlio
Module Contents
Functions
|
Gets information from an example file |
|
Gets count of dataset |
|
Gets train and validation indices of dataset |
|
Gets train and validation data for cross validation. |
|
Splits dataset object into smaller chunks |
|
Checks if example file has scaleror not from info |
|
|
|
|
|
Gets training data from file name |
|
Gets training data from file info |
|
|
|
|
|
|
|
Saves trained model for any platform workflow |
|
Get model and model information |
|
Gets model apply info from file name |
|
Gets model apply info from example file info |
|
|
|
|
|
|
|
Gets model type |
|
|
Attributes
- dgbpy.mlio.nladbdirid = 100060
- dgbpy.mlio.mlinpgrp = Deep Learning Example Data
- dgbpy.mlio.mltrlgrp = Deep Learning Model
- dgbpy.mlio.dgbtrl = dGB
- dgbpy.mlio.getInfo(filenm, quick=False)
Gets information from an example file
- Parameters:
filenm (str): ffile name/path in hdf5 format
- quick (bool): when set to True, info is gottenn quickly leaving out some info(e.g. datasets),
defaults to False and loads all informaton
- Returns:
dict: information from data file or model file
- dgbpy.mlio.datasetCount(dsets)
Gets count of dataset
- Parameters:
dsets (dict): dataset
- Returns:
dict: counts for target attribute(s) or well(s) for project
- dgbpy.mlio.getDatasetNms(dsets, validation_split=None, valid_inputs=None)
Gets train and validation indices of dataset
- Parameters:
dsets (dict): dataset
validation_split (float): size of validation data (between 0-1)
valid_inputs (iter):
- Returns:
dict: train and validation indices
- dgbpy.mlio.getCrossValidationIndices(dsets, seed=None, valid_inputs=1, nbfolds=5)
Gets train and validation data for cross validation.
- Parameters:
dsets (dict): dictionary of survey names and datasets
n_wells (int): number of wells to use as the validat ion set
- Returns:
list: list of dictionaries containing train and validation data for each fold
- dgbpy.mlio.getChunks(dsets, nbchunks)
Splits dataset object into smaller chunks
- Parameters:
dsets (dict): dataset
nbchunks (int): number of data chunks to be created
- Returns:
dict: chunks from dataset stored as dictionaries
- dgbpy.mlio.hasScaler(infos, inputsel=None)
Checks if example file has scaleror not from info
- Parameters:
infos (dict): information about example file
inputsel (bool or NoneType):
- Returns:
bool: True if dataset info has scaler key, False if other
- dgbpy.mlio.getDatasetsByGroup(dslist, groupnm)
- dgbpy.mlio.getSomeDatasets(dslist, decim=None)
- dgbpy.mlio.getTrainingData(filenm, decim=False)
Gets training data from file name
- Parameters:
filenm (str): path to file in hdf5 format
decim (bool):
- Returns:
dict: train, validation datasets as arrays, and info on example file
- dgbpy.mlio.getTrainingDataByInfo(info, dsetsel=None)
Gets training data from file info
- Parameters:
info (dict): information about example file
dsetsel ():
- Returns:
dict: train, validation datasets as arrays, and info on example file
- dgbpy.mlio.getClasses(info, y_vectors)
- dgbpy.mlio.normalize_class_vector(arr, classes)
- dgbpy.mlio.unnormalize_class_vector(arr, classes)
- dgbpy.mlio.saveModel(model, inpfnm, platform, infos, outfnm)
Saves trained model for any platform workflow
- Parameters:
model (obj): trained model on any platform
inpfnm (str): example file name in hdf5 format
platform (str): machine learning platform (options; keras, Scikit-learn, torch)
infos (dict): example file info
outfnm (str): name of model to be saved
- dgbpy.mlio.getModel(modelfnm, fortrain=False, pars=None)
Get model and model information
- Parameters:
modelfnm (str): model file path/name in hdf5 format
fortrain (bool): specifies if the model might be further trained
pars (dict): parameters to be used when restoring the model if needed
- Returs:
tuple: (trained model and model/project info)
- dgbpy.mlio.getApplyInfoFromFile(modelfnm, outsubsel=None)
Gets model apply info from file name
- Parameters:
modelfnm (str): model file path/name in hdf5 format
outsubsel ():
- Returns:
dict: apply information
- dgbpy.mlio.getApplyInfo(infos, outsubsel=None)
Gets model apply info from example file info
- Parameters:
infos (dict): example file info
outsubsel ():
- Returns:
dict: apply information
- dgbpy.mlio.dblistall
- dgbpy.mlio.modelNameIsFree(modnm, type, args, reload=True)
- dgbpy.mlio.modelNameExists(modnm, type, args, reload=True)
- dgbpy.mlio.dbInfoForModel(modnm, args, reload=True)
- dgbpy.mlio.getModelType(infos)
Gets model type
- Parameters:
infos (dict): example file info
- Returns:
str: Type ofmodel/workflow
- dgbpy.mlio.getSaveLoc(outnm, ftype, args)
- dgbpy.mlio.announceShowTensorboard()
- dgbpy.mlio.announceTrainingFailure()
- dgbpy.mlio.announceTrainingSuccess()