python.dgbpy.mlio

Module Contents

Functions

getInfo(filenm, quick=False)

Gets information from an example file

datasetCount(dsets)

Gets count of dataset

getDatasetNms(dsets, validation_split=None, valid_inputs=None)

Gets train and validation indices of dataset

getChunks(dsets, nbchunks)

Splits dataset object into smaller chunks

hasScaler(infos, inputsel=None)

Checks if example file has scaleror not from info

getDatasetsByGroup(dslist, groupnm)

getSomeDatasets(dslist, decim=None)

getTrainingData(filenm, decim=False)

Gets training data from file name

getTrainingDataByInfo(info, dsetsel=None)

Gets training data from file info

getClasses(info, y_vectors)

normalize_class_vector(arr, classes)

unnormalize_class_vector(arr, classes)

saveModel(model, inpfnm, platform, infos, outfnm)

Saves trained model for any platform workflow

getModel(modelfnm, fortrain=False, pars=None)

Get model and model information

getApplyInfoFromFile(modelfnm, outsubsel=None)

Gets model apply info from file name

getApplyInfo(infos, outsubsel=None)

Gets model apply info from example file info

modelNameIsFree(modnm, type, args, reload=True)

modelNameExists(modnm, type, args, reload=True)

dbInfoForModel(modnm, args, reload=True)

getModelType(infos)

Gets model type

getSaveLoc(outnm, ftype, args)

Attributes

nladbdirid

mlinpgrp

mltrlgrp

dgbtrl

dblistall

python.dgbpy.mlio.nladbdirid = 100060
python.dgbpy.mlio.mlinpgrp = Deep Learning Example Data
python.dgbpy.mlio.mltrlgrp = Deep Learning Model
python.dgbpy.mlio.dgbtrl = dGB
python.dgbpy.mlio.getInfo(filenm, quick=False)

Gets information from an example file

Parameters:

  • filenm (str): ffile name/path in hdf5 format

  • quick (bool): when set to True, info is gottenn quickly leaving out some info(e.g. datasets),

    defaults to False and loads all informaton

Returns:

  • dict: information from data file or model file

python.dgbpy.mlio.datasetCount(dsets)

Gets count of dataset

Parameters:

  • dsets (dict): dataset

Returns:

  • dict: counts for target attribute(s) or well(s) for project

python.dgbpy.mlio.getDatasetNms(dsets, validation_split=None, valid_inputs=None)

Gets train and validation indices of dataset

Parameters:

  • dsets (dict): dataset

  • validation_split (float): size of validation data (between 0-1)

  • valid_inputs (iter):

Returns:

  • dict: train and validation indices

python.dgbpy.mlio.getChunks(dsets, nbchunks)

Splits dataset object into smaller chunks

Parameters:

  • dsets (dict): dataset

  • nbchunks (int): number of data chunks to be created

Returns:

  • dict: chunks from dataset stored as dictionaries

python.dgbpy.mlio.hasScaler(infos, inputsel=None)

Checks if example file has scaleror not from info

Parameters:

  • infos (dict): information about example file

  • inputsel (bool or NoneType):

Returns:

bool: True if dataset info has scaler key, False if other

python.dgbpy.mlio.getDatasetsByGroup(dslist, groupnm)
python.dgbpy.mlio.getSomeDatasets(dslist, decim=None)
python.dgbpy.mlio.getTrainingData(filenm, decim=False)

Gets training data from file name

Parameters:

  • filenm (str): path to file in hdf5 format

  • decim (bool):

Returns:

  • dict: train, validation datasets as arrays, and info on example file

python.dgbpy.mlio.getTrainingDataByInfo(info, dsetsel=None)

Gets training data from file info

Parameters:

  • info (dict): information about example file

  • dsetsel ():

Returns:

  • dict: train, validation datasets as arrays, and info on example file

python.dgbpy.mlio.getClasses(info, y_vectors)
python.dgbpy.mlio.normalize_class_vector(arr, classes)
python.dgbpy.mlio.unnormalize_class_vector(arr, classes)
python.dgbpy.mlio.saveModel(model, inpfnm, platform, infos, outfnm)

Saves trained model for any platform workflow

Parameters:

  • model (obj): trained model on any platform

  • inpfnm (str): example file name in hdf5 format

  • platform (str): machine learning platform (options; keras, Scikit-learn, torch)

  • infos (dict): example file info

  • outfnm (str): name of model to be saved

python.dgbpy.mlio.getModel(modelfnm, fortrain=False, pars=None)

Get model and model information

Parameters:

  • modelfnm (str): model file path/name in hdf5 format

  • fortrain (bool): specifies if the model might be further trained

  • pars (dict): parameters to be used when restoring the model if needed

Returs:

  • tuple: (trained model and model/project info)

python.dgbpy.mlio.getApplyInfoFromFile(modelfnm, outsubsel=None)

Gets model apply info from file name

Parameters:

  • modelfnm (str): model file path/name in hdf5 format

  • outsubsel ():

Returns:

  • dict: apply information

python.dgbpy.mlio.getApplyInfo(infos, outsubsel=None)

Gets model apply info from example file info

Parameters:

  • infos (dict): example file info

  • outsubsel ():

Returns:

  • dict: apply information

python.dgbpy.mlio.dblistall
python.dgbpy.mlio.modelNameIsFree(modnm, type, args, reload=True)
python.dgbpy.mlio.modelNameExists(modnm, type, args, reload=True)
python.dgbpy.mlio.dbInfoForModel(modnm, args, reload=True)
python.dgbpy.mlio.getModelType(infos)

Gets model type

Parameters:

  • infos (dict): example file info

Returns:

  • str: Type ofmodel/workflow

python.dgbpy.mlio.getSaveLoc(outnm, ftype, args)