The datamanager wraps a dataloader and combines it with augmentations and multiprocessing.


class BaseDataManager(data, batch_size, n_process_augmentation, transforms, sampler_cls=<class 'delira.data_loading.sampler.sequential_sampler.SequentialSampler'>, sampler_kwargs={}, data_loader_cls=None, dataset_cls=None, load_fn=<function default_load_fn_2d>, from_disc=True, **kwargs)[source]

Bases: object

Class to Handle Data Creates Dataset , Dataloader and BatchGenerator


Property to access the batchsize

Returns:the batchsize
Return type:int

Property to access the current data loader class

Returns:Subclass of SlimDataLoaderBase
Return type:type

Property to access the current dataset

Returns:the current dataset
Return type:AbstractDataset

Create DataLoader and Batchgenerator

Parameters:seed (int) – seed for Random Number Generator
Return type:MultiThreadedAugmenter
Raises:AssertionErrorBaseDataManager.n_batches is smaller than or equal to zero

Returns a Subset of the current datamanager based on given indices

Parameters:indices (iterable) – valid indices to extract subset from current dataset
Returns:manager containing the subset
Return type:BaseDataManager

Returns Number of Batches based on batchsize, number of samples and number of processes

Returns:Number of Batches
Return type:int
Raises:AssertionErrorBaseDataManager.n_samples is smaller than or equal to zero

Property to access the number of augmentation processes

Returns:number of augmentation processes
Return type:int

Number of Samples

Returns:Number of Samples
Return type:int

Property to access the current sampler

Returns:the current sampler
Return type:AbstractSampler
train_test_split(*args, **kwargs)[source]

Calls :method:`AbstractDataset.train_test_split` and returns a manager for each subset with same configuration as current manager

  • *args – positional arguments for sklearn.model_selection.train_test_split
  • **kwargs – keyword arguments for sklearn.model_selection.train_test_split

Property to access the current data transforms

Returns:The transformation, can either be None or an instance of AbstractTransform
Return type:None, AbstractTransform
update_state_from_dict(new_state: dict)[source]

Updates internal state and therfore the behavior from dict. If a key is not specified, the old attribute value will be used

Parameters:new_state (dict) –

The dict to update the state from. Valid keys are:

  • batch_size
  • n_process_augmentation
  • data_loader_cls
  • sampler
  • sampling_kwargs
  • transforms

If a key is not specified, the old value of the corresponding attribute will be used

Raises:KeyError – Invalid keys are specified