Datamanager¶

The datamanager wraps a dataloader and combines it with augmentations and multiprocessing.

BaseDataManager¶

class BaseDataManager(data, batch_size, n_process_augmentation, transforms, sampler_cls=<class 'delira.data_loading.sampler.sequential_sampler.SequentialSampler'>, sampler_kwargs={}, data_loader_cls=None, dataset_cls=None, load_fn=<function default_load_fn_2d>, from_disc=True, **kwargs)[source]¶

Bases: object

Class to Handle Data Creates Dataset , Dataloader and BatchGenerator

batch_size¶

Property to access the batchsize

Returns:	the batchsize
Return type:	int

data_loader_cls¶

Property to access the current data loader class

Returns:	Subclass of `SlimDataLoaderBase`
Return type:	type

dataset¶

Property to access the current dataset

Returns:	the current dataset
Return type:	`AbstractDataset`

get_batchgen(seed=1)[source]¶

Create DataLoader and Batchgenerator

Parameters:	seed (int) – seed for Random Number Generator
Returns:	Batchgenerator
Return type:	MultiThreadedAugmenter
Raises:	`AssertionError` – `BaseDataManager.n_batches` is smaller than or equal to zero

get_subset(indices)[source]¶

Returns a Subset of the current datamanager based on given indices

Parameters:	indices (iterable) – valid indices to extract subset from current dataset
Returns:	manager containing the subset
Return type:	`BaseDataManager`

n_batches¶

Returns Number of Batches based on batchsize, number of samples and number of processes

Returns:	Number of Batches
Return type:	int
Raises:	`AssertionError` – `BaseDataManager.n_samples` is smaller than or equal to zero

n_process_augmentation¶

Property to access the number of augmentation processes

Returns:	number of augmentation processes
Return type:	int

n_samples¶

Number of Samples

Returns:	Number of Samples
Return type:	int

sampler¶

Property to access the current sampler

Returns:	the current sampler
Return type:	`AbstractSampler`

train_test_split(*args, **kwargs)[source]¶

Calls :method:`AbstractDataset.train_test_split` and returns a manager for each subset with same configuration as current manager

Parameters:	args – positional arguments for `sklearn.model_selection.train_test_split` *kwargs – keyword arguments for `sklearn.model_selection.train_test_split`

transforms¶

Property to access the current data transforms

Returns:	The transformation, can either be None or an instance of `AbstractTransform`
Return type:	None, `AbstractTransform`

update_state_from_dict(new_state: dict)[source]¶

Updates internal state and therfore the behavior from dict. If a key is not specified, the old attribute value will be used

Parameters:

new_state (dict) –

The dict to update the state from. Valid keys are:

batch_size

n_process_augmentation

data_loader_cls

sampler

sampling_kwargs

transforms

If a key is not specified, the old value of the corresponding attribute will be used

Raises: KeyError – Invalid keys are specified