Callbacks

Callbacks are essential to provide a uniform API for tasks like early stopping etc. The PyTorch learning rate schedulers are also implemented as callbacks. Every callback should ber derived from AbstractCallback and must provide the methods at_epoch_begin and at_epoch_end.

AbstractCallback

class AbstractCallback(*args, **kwargs)[source]

Bases: object

Implements abstract callback interface. All callbacks should be derived from this class

See also

AbstractNetworkTrainer

at_epoch_begin(trainer, **kwargs)[source]

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)[source]

Function which will be executed at end of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

EarlyStopping

class EarlyStopping(monitor_key, min_delta=0, patience=0, mode='min')[source]

Bases: delira.training.callbacks.abstract_callback.AbstractCallback

Implements Early Stopping as callback

See also

AbstractCallback

_is_better(metric)[source]

Helper function to decide whether the current metric is better than the best metric so far

Parameters

metric – current metric value

Returns

whether this metric is the new best metric or not

Return type

bool

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)[source]

Actual early stopping: Checks at end of each epoch if monitored metric is new best and if it hasn’t improved over self.patience epochs, the training will be stopped

Parameters
  • trainer (AbstractNetworkTrainer) – the trainer whose arguments can be modified

  • **kwargs – additional keyword arguments

Returns

trainer with modified attributes

Return type

AbstractNetworkTrainer

DefaultPyTorchSchedulerCallback

class DefaultPyTorchSchedulerCallback(*args, **kwargs)[source]

Bases: delira.training.callbacks.abstract_callback.AbstractCallback

Implements a Callback, which at_epoch_end function is suitable for most schedulers

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)[source]

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

CosineAnnealingLRCallback

class CosineAnnealingLRCallback(optimizer, T_max, eta_min=0, last_epoch=-1)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s CosineAnnealingLR Scheduler as callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

ExponentialLRCallback

class ExponentialLRCallback(optimizer, gamma, last_epoch=-1)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s ExponentialLR Scheduler as callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

LambdaLRCallback

class LambdaLRCallback(optimizer, lr_lambda, last_epoch=-1)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s LambdaLR Scheduler as callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

MultiStepLRCallback

class MultiStepLRCallback(optimizer, milestones, gamma=0.1, last_epoch=-1)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s MultiStepLR Scheduler as callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

ReduceLROnPlateauCallback

class ReduceLROnPlateauCallback(optimizer, mode='min', factor=0.1, patience=10, verbose=False, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0, eps=1e-08)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s ReduceLROnPlateau Scheduler as Callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)[source]

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

StepLRCallback

class StepLRCallback(optimizer, step_size, gamma=0.1, last_epoch=-1)[source]

Bases: delira.training.callbacks.pytorch_schedulers.DefaultPyTorchSchedulerCallback

Wraps PyTorch’s StepLR Scheduler as callback

at_epoch_begin(trainer, **kwargs)

Function which will be executed at begin of each epoch

Parameters
  • trainer (AbstractNetworkTrainer) –

  • **kwargs – additional keyword arguments

Returns

modified trainer attributes, where the name must correspond to the trainer’s attribute name

Return type

dict

at_epoch_end(trainer, **kwargs)

Executes a single scheduling step

Parameters
  • trainer (PyTorchNetworkTrainer) – the trainer class, which can be changed

  • **kwargs – additional keyword arguments

Returns

modified trainer

Return type

PyTorchNetworkTrainer

CosineAnnealingLRCallbackPyTorch

CosineAnnealingLRCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.CosineAnnealingLRCallback

ExponentialLRCallbackPyTorch

ExponentialLRCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.ExponentialLRCallback

LambdaLRCallbackPyTorch

LambdaLRCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.LambdaLRCallback

MultiStepLRCallbackPyTorch

MultiStepLRCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.MultiStepLRCallback

ReduceLROnPlateauCallbackPyTorch

ReduceLROnPlateauCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.ReduceLROnPlateauCallback

StepLRCallbackPyTorch

StepLRCallbackPyTorch

alias of delira.training.callbacks.pytorch_schedulers.StepLRCallback