Datamanager¶

The datamanager wraps a dataloader and combines it with augmentations and multiprocessing.

BaseDataManager¶

class BaseDataManager(data, batch_size, n_process_augmentation, transforms, sampler_cls=<class 'delira.data_loading.sampler.sequential_sampler.SequentialSampler'>, sampler_kwargs={}, data_loader_cls=None, dataset_cls=None, load_fn=<function default_load_fn_2d>, from_disc=True, **kwargs)[source]¶

Bases: object

Class to Handle Data Creates Dataset , Dataloader and BatchGenerator

batch_size¶

Property to access the batchsize

Returns: the batchsize
Return type: int

data_loader_cls¶

Property to access the current data loader class

Returns: Subclass of SlimDataLoaderBase
Return type: type

dataset¶

Property to access the current dataset

Returns: the current dataset
Return type: AbstractDataset

get_batchgen(seed=1)[source]¶

Create DataLoader and Batchgenerator

Parameters: seed (int) – seed for Random Number Generator
Returns: Batchgenerator
Return type: MultiThreadedAugmenter
Raises: AssertionError – BaseDataManager.n_batches is smaller than or equal to zero

get_subset(indices)[source]¶

Returns a Subset of the current datamanager based on given indices

Parameters: indices (iterable) – valid indices to extract subset from current dataset
Returns: manager containing the subset
Return type: BaseDataManager

n_batches¶

Returns Number of Batches based on batchsize, number of samples and number of processes

Returns: Number of Batches
Return type: int
Raises: AssertionError – BaseDataManager.n_samples is smaller than or equal to zero

n_process_augmentation¶

Property to access the number of augmentation processes

Returns: number of augmentation processes
Return type: int

n_samples¶

Number of Samples

Returns: Number of Samples
Return type: int

sampler¶

Property to access the current sampler

Returns: the current sampler
Return type: AbstractSampler

train_test_split(*args, **kwargs)[source]¶

Calls :method:`AbstractDataset.train_test_split` and returns a manager for each subset with same configuration as current manager

Parameters

*args – positional arguments for sklearn.model_selection.train_test_split
**kwargs – keyword arguments for sklearn.model_selection.train_test_split

transforms¶

Property to access the current data transforms

Returns: The transformation, can either be None or an instance of AbstractTransform
Return type: None, AbstractTransform

update_state_from_dict(new_state: dict)[source]¶

Updates internal state and therfore the behavior from dict. If a key is not specified, the old attribute value will be used

Parameters

new_state (dict) –

The dict to update the state from. Valid keys are:

batch_size

n_process_augmentation

data_loader_cls

sampler

sampling_kwargs

transforms

If a key is not specified, the old value of the corresponding attribute will be used

Raises

KeyError – Invalid keys are specified