Datamanager

The datamanager wraps a dataloader and combines it with augmentations and multiprocessing.

BaseDataManager

class BaseDataManager(data, batch_size, n_process_augmentation, transforms, sampler_cls=<class 'delira.data_loading.sampler.sequential_sampler.SequentialSampler'>, sampler_kwargs={}, data_loader_cls=None, dataset_cls=None, load_fn=<function default_load_fn_2d>, from_disc=True, **kwargs)

Bases: object

Class to Handle Data Creates Dataset , Dataloader and BatchGenerator

property batch_size

Property to access the batchsize

Returns

the batchsize

Return type

int

property data_loader_cls

Property to access the current data loader class

Returns

Subclass of SlimDataLoaderBase

Return type

type

property dataset

Property to access the current dataset

Returns

the current dataset

Return type

AbstractDataset

get_batchgen(seed=1)

Create DataLoader and Batchgenerator

Parameters

seed (int) – seed for Random Number Generator

Returns

Batchgenerator

Return type

Augmenter

Raises

AssertionError – BaseDataManager.n_batches is smaller than or equal to zero

get_subset(indices)

Returns a Subset of the current datamanager based on given indices

Parameters

indices (iterable) – valid indices to extract subset from current dataset

Returns

manager containing the subset

Return type

BaseDataManager

property n_batches

Returns Number of Batches based on batchsize and number of samples

Returns

Number of Batches

Return type

int

Raises

AssertionError – BaseDataManager.n_samples is smaller than or equal to zero

property n_process_augmentation

Property to access the number of augmentation processes

Returns

number of augmentation processes

Return type

int

property n_samples

Number of Samples

Returns

Number of Samples

Return type

int

property sampler

Property to access the current sampler

Returns

the current sampler

Return type

AbstractSampler

train_test_split(*args, **kwargs)

Calls :method:`AbstractDataset.train_test_split` and returns a manager for each subset with same configuration as current manager

Parameters

• *args – positional arguments for sklearn.model_selection.train_test_split

• **kwargs – keyword arguments for sklearn.model_selection.train_test_split

property transforms

Property to access the current data transforms

Returns

The transformation, can either be None or an instance of AbstractTransform

Return type

None, AbstractTransform

update_state_from_dict(new_state: dict)

Updates internal state and therfore the behavior from dict. If a key is not specified, the old attribute value will be used

Parameters

new_state (dict) –

The dict to update the state from. Valid keys are:

• batch_size

• n_process_augmentation

• data_loader_cls

• sampler

• sampling_kwargs

• transforms

If a key is not specified, the old value of the corresponding attribute will be used

Raises

KeyError – Invalid keys are specified