utils package

utils.dataLoader module

Dataset and Dataloader classes for both numerical test cases

class utils.dataLoader.BackwardStepLoader(training_dir, testing_dir, shuffle=True, log=None)

Bases: utils.dataLoader.TMGLowDataLoader

Class used for creating data loaders for the backwards step numerical example Args:

ntrain (int): number of training data ntest (int): number of testing data data_dir (string): path of numpy data files shuffle (boolean): shuffle the training data or not log (Log): logging class

createTestingLoader(ntest, u0, inUpscale=1, batch_size=32)

Creates the testing loader Args:

ntrain (np.array): Numpy array of training-indexes u0 (np.array): Input velocity for each training index (used for normalizing) inUpscale (int): Initial upscaling, used to just make the architecture similar batch_size (int): Training batch-size

createTrainingLoader(ntrain, u0, tSplit=1, inUpscale=1, batch_size=32, tar_noise_std=0)

Creates the training loader Args:

ntrain (np.array): Numpy array of training-indexes u0 (np.array): Input velocity for each training index (used for normalizing) tSplit (int): Number of time to split the simulation data into smaller time-series for training inUpscale (int): Initial upscaling, used to just make the architecture similar batch_size (int): Training batch-size tar_noise_std (int): Random noise to add on the target

class utils.dataLoader.CylinderArrayLoader(training_dir, testing_dir, shuffle=True, log=None)

Bases: utils.dataLoader.TMGLowDataLoader

Class used for creating data loaders for the cylinder array numerical example Args:

ntrain (int): number of training data ntest (int): number of testing data data_dir (string): path of numpy data files shuffle (boolean): shuffle the training data or not log (Log): logging class

createTestingLoader(ntest, inUpscale=1, batch_size=32)

Creates the testing loader Args:

ntrain (np.array): Numpy array of training-indexes u0 (np.array): Input velocity for each training index (used for normalizing) inUpscale (int): Initial upscaling, used to just make the architecture similar batch_size (int): Training batch-size

createTrainingLoader(ntrain, tSplit=1, inUpscale=1, batch_size=32, tar_noise_std=0)

Creates the training loader Args:

ntrain (np.array): Numpy array of training-indexes u0 (np.array): Input velocity for each training index (used for normalizing) tSplit (int): Number of time to split the simulation data into smaller time-series for training inUpscale (int): Initial upscaling, used to just make the architecture similar batch_size (int): Training batch-size tar_noise_std (int): Random noise to add on the target

class utils.dataLoader.DataLoaderAuto

Bases: object

classmethod init_data_loaders(args, model, log)

Used to initialize data-loaders for each numerical test case Args:

classmethod setupBackwardStepLoaders(args, model, log)
classmethod setupCylinderLoaders(args, model, log)
class utils.dataLoader.TMGLowDataLoader(training_dir='.', testing_dir='.', log=None)

Bases: object

Parent class for TM-Glow dataloader creators Note: These are not used as actual data loaders, they create data loaders

calcNormalizingParams(inputData, targetData)

Calculates the hyper-paramters used for normalizing the training input/output data. Normalizes data to a standard unit Gaussian. Args:

inputData (tensor): [b,t,c,d1,d2] tensor of low-fidelity inputs targetData (tensor): [b,t,c,d1*,d2*] tensor of high-fidelity target

normalizeInputData(inputData)

Normalize the input tensor on each channel (x-vel, y-vel, pressure)

normalizeTargetData(targetData)

Normalize the target tensor on each channel (x-vel, y-vel, pressure)

readFluidData(input_file_name: str, target_file_name: str, fStride=1, cStride=1)
setNormalizingParams(model)

Given a PyTorch model this sets the normalizing paramters of the loader class using what is stored in the model. This is done to save normalizing constants between runs. Args:

model: PyTorch model with normalizing constants as

transferNormalizingParams(model)

Given a PyTorch model this gets the calculated normalizing parameters and assigned them to registered parameters of the model. This is done to save normalizing constants between runs. Args:

model: PyTorch model with normalizing constants params to be set device (PyTorch device): device the PyTorch model is on

class utils.dataLoader.TrainingDataset(inputs, targets, lstm_seeds, input_noise_std=0.01, tar_noise_std=0.0)

Bases: torch.utils.data.dataset.Dataset

Training data-set with additive noise option for targets

utils.log module

Logging utilities, used for colored outputs and saving training/testing data to log files.

class utils.log.Log(args=None, file_base='output', record=True)

Bases: object

Class for logging outputs with colors and timestamps

error(str0)
info(str0)
log(str0, rec=True)
logLoss(epoch, loss, trainingMNLL, trainingMSE)
logTest(epoch, testMNLL, testMSE)
print_progress(iteration, total, prefix='[15:05:26]', suffix='', decimals=1, bar_length=50)

Call in a loop to create terminal progress bar @params:

iteration - Required : current iteration (Int) total - Required : total iterations (Int) prefix - Optional : prefix string (Str) suffix - Optional : suffix string (Str) decimals - Optional : positive number of decimals in percent complete (Int) bar_length - Optional : character length of bar (Int)

success(str0)
warning(str0)
class utils.log.bcolors

Bases: object

BOLD = '\x1b[1m'
ENDC = '\x1b[0m'
FAIL = '\x1b[91m'
HEADER = '\x1b[95m'
OKBLUE = '\x1b[94m'
OKGREEN = '\x1b[92m'
UNDERLINE = '\x1b[4m'
WARNING = '\x1b[93m'
class utils.log.uid

Bases: object

moment = '2021-Jan-13_15_05_26'

utils.parallel module

Utilities for training TM-Glow in parallel as well as calculating the loss in parallel on different GPUs for memory purposes.

Original Implementation by Zhang, Rutgers University https://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255

class utils.parallel.DataParallelCriterion(module, device_ids=None, output_device=None, dim=0)

Bases: torch.nn.parallel.data_parallel.DataParallel

Calculate loss in multiple-GPUs, which balance the memory usage. The targets are splitted across the specified devices by chunking in the batch dimension. Reference:

Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, Amit Agrawal. “Context Encoding for Semantic Segmentation. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018

Example::
>>> net = encoding.nn.DataParallelModel(model, device_ids=[0, 1, 2])
>>> criterion = encoding.nn.DataParallelCriterion(criterion, device_ids=[0, 1, 2])
>>> y = net(x)
>>> loss = criterion(y, target)
forward(inputs, *targets, **kwargs)
training: bool
utils.parallel.allreduce(*inputs)

Cross GPU all reduce autograd operation for calculate mean and variance in SyncBN.

utils.utils module

Random utilities

utils.utils.getGpuMemoryMap()

Get the current gpu usage.

Returns:

usage: dict Keys are device ids as integers. Values are memory usage as integers in MB.

utils.utils.loadWorkspace(args, file_dir, file_name='nsWorkspace', file_id=0)

Loads parameters of the model, optimizer, scheduler and program arguements from zip file Args:

file_dir (string): Directory of workspace zip file file_name (string): File name of workspace (default: nsWorkspace) file_id (int): Current epoch or desired ID number for saved workspace

Returns:

args (argparse): object with programs arguements model_state_dict (dict): PyTorch state dictionary of model parameters optimizer_state_dict (dict): PyTorch state dictionary of optimizers parameters

utils.utils.modelPred(args, model, testing_loader, log, samples=1, stride=1, tmax=1)

Tests the model on a single GPUs, used for plotting Args:

model (torch.nn.Module): PyTorch model to test stride (int): The stride the low-fidelity input takes compared to output samples (int): Number of prediction to sample from the model

Returns:

ypred (torch.Tensor): tensor of un-normalized predicted values ytarget (torch.Tensor): tensor of un-normalized target values of test cases yinput (torch.Tensor): tensor of un-normalized input values

utils.utils.saveWorkspace(args, model, optimizer, file_name='nsWorkspace', file_id=0)

Saves parameters of the model, optimizer, scheduler and program arguements into a zip file Args:

args (argparse): object with programs arguements model: Neural network model optimizer: PyTorch optimizer file_id (int): Current epoch or desired ID number for saved workspace

utils.utils.toNumpy(tensor)

Converts Pytorch tensor to numpy array

utils.utils.toTuple(a)

Converts array to tuple

utils.viz module

Visualization functions used during training

utils.viz.plotNumericalPred(args, yPred0, yTarget0, nut0, bidx=0, tstep=0, epoch=0)

Used to plot predictions of the model during training Args:

yPred0 (torch.Tensor): [samp x mb x t x w x h] model prediction uTarget (torch.Tenosr): Target

utils.viz.plotVelocityPred(args, input0, yPred0, yTarget0, bidx=0, stride=1, epoch=0)

Used to plot predictions of the model during training Args:

yPred0 (torch.Tensor): [samp x mb x t x w x h] model prediction uTarget (torch.Tenosr): Target