Sampler

Sampler define the way of iterating over the dataset and returning samples.

AbstractSampler

class AbstractSampler(indices=None)[source]

Bases: object

Class to define an abstract Sampling API

_check_batchsize(n_indices)[source]

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

abstract _get_indices(n_indices)[source]

Function to return a specific number of indices. Implements the actual sampling strategy.

Parameters

n_indices (int) – Number of indices to return

Returns

List with sampled indices

Return type

list

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

LambdaSampler

class LambdaSampler(indices, sampling_fn)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Arbitrary Sampling methods specified by a function which takes the index_list and the number of indices to return

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

Returns

list of sampled indices

Return type

list

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

RandomSampler

class RandomSampler(indices)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Random Sampling With Replacement from whole Dataset

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

Returns

list of sampled indices

Return type

list

Raises

StopIteration – If maximal number of samples is reached

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

PrevalenceRandomSampler

class PrevalenceRandomSampler(indices, shuffle_batch=True)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements random Per-Class Sampling and ensures same number of samplers per batch for each class

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

Returns

list of sampled indices

Return type

list

Raises

StopIteration – If maximal number of samples is reached

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialized sampler

Return type

AbstractSampler

StoppingPrevalenceRandomSampler

class StoppingPrevalenceRandomSampler(indices, shuffle_batch=True)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements random Per-Class Sampling and ensures same number of samplers per batch for each class; Stops if out of samples for smallest class

_check_batchsize(n_indices)[source]

Checks if batchsize is valid for all classes

Parameters

n_indices (int) – the number of samples to return

Returns

number of samples per class to return

Return type

dict

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

Raises

StopIteration – If end of class indices is reached for one class:

Returns

list

Return type

list of sampled indices

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialized sampler

Return type

AbstractSampler

SequentialSampler

class SequentialSampler(indices)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Sequential Sampling from whole Dataset

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

:raises StopIteration : If end of dataset reached:

Returns

list of sampled indices

Return type

list

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

PrevalenceSequentialSampler

class PrevalenceSequentialSampler(indices, shuffle_batch=True)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Per-Class Sequential sampling and ensures same number of samples per batch for each class; If out of samples for one class: restart at first sample

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

:raises StopIteration : If end of class indices is reached:

Returns

list of sampled indices

Return type

list

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

StoppingPrevalenceSequentialSampler

class StoppingPrevalenceSequentialSampler(indices, shuffle_batch=True)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Per-Class Sequential sampling and ensures same number of samples per batch for each class; Stops if all samples of first class have been sampled

_check_batchsize(n_indices)[source]

Checks if batchsize is valid for all classes

Parameters

n_indices (int) – the number of samples to return

Returns

number of samples per class to return

Return type

dict

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

:raises StopIteration : If end of class indices is reached for one class:

Returns

list of sampled indices

Return type

list

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler

WeightedRandomSampler

class WeightedRandomSampler(indices, weights=None)[source]

Bases: delira.data_loading.sampler.abstract_sampler.AbstractSampler

Implements Weighted Random Sampling

_check_batchsize(n_indices)

Checks if the batchsize is valid (and truncates batches if necessary). Will also raise StopIteration if enough batches sampled

Parameters

n_indices (int) – number of indices to sample

Returns

number of indices to sample (truncated if necessary)

Return type

int

Raises

StopIteration – if enough batches sampled

_get_indices(n_indices)[source]

Actual Sampling

Parameters

n_indices (int) – number of indices to return

Returns

list of sampled indices

Return type

list

Raises
  • StopIteration – If maximal number of samples is reached

  • ValueError – if weights or cum_weights don’t match the population

classmethod from_dataset(dataset: delira.data_loading.dataset.AbstractDataset, **kwargs)[source]

Classmethod to initialize the sampler from a given dataset

Parameters

dataset (AbstractDataset) – the given dataset

Returns

The initialzed sampler

Return type

AbstractSampler