kgcnn.training package¶
Submodules¶
kgcnn.training.callbacks module¶
kgcnn.training.history module¶
-
kgcnn.training.history.load_time_list(file_path, folds)¶
-
kgcnn.training.history.save_history_score(histories: list, filepath: Optional[str] = None, loss_name: Optional[str] = None, val_loss_name: Optional[str] = None, data_unit: str = '', model_name: str = '', file_name: str = 'score.yaml', model_version: str = '', dataset_name: str = '', model_class: str = '', execute_folds: Optional[Union[list, int]] = None, multi_target_indices: Optional[Union[list, int]] = None, trajectory_name: Optional[str] = None, seed: Optional[int] = None, time_list: Optional[list] = None)[source]¶ Save fit results from fit histories to file.
This function is used in training scripts to record final training and validation metrics.
- Parameters
histories (list) – List of
tf.keras.callbacks.History()objects.filepath (str) – Full path where to save plot to, without the name of the file. Default is “”.
loss_name (str) – Which loss or metric to pick from history. Default is “loss”.
val_loss_name (str) – Which validation loss or metric to pick from history. Default is “val_loss”.
data_unit (str) – Unit of the loss. Default is “”.
model_name (str) – Name of the model. Default is “”.
file_name (str) – File name base. Model name and dataset will be added to the name. Default is “”.
model_version (str) – Version of the model. Default is “”.
dataset_name (str) – Name of the dataset which was fitted to. Default is “”.
model_class (str) – Model class or generator. Default is “”.
multi_target_indices (list) – List of indices for multi target training. Default is None.
trajectory_name (str) – Name of the trajectory if known. Default is None.
seed (int) – Random seed to log. Default is None.
time_list (list) – List of training time info.
- Returns
Score which was saved to file.
- Return type
kgcnn.training.hyper module¶
-
class
kgcnn.training.hyper.HyperParameter(hyper_info: Union[str, dict], hyper_category: Optional[str] = None, model_name: Optional[str] = None, model_module: Optional[str] = None, model_class: str = 'make_model', dataset_name: Optional[str] = None, dataset_class: Optional[str] = None, dataset_module: Optional[str] = None)[source]¶ Bases:
objectA class to store hyperparameter for a specific dataset and model, exposing them for model training scripts.
This includes training parameters and a set of general information like a path for output of the training stats or the expected version of
kgcnn. The class methods will extract and possibly serialize or deserialize the necessary kwargs from the hyperparameter dictionary.hyper = HyperParameter(hyper_info={"model": {"config":{}}, "training": {}, "data":{"dataset":{}}})
-
__init__(hyper_info: Union[str, dict], hyper_category: Optional[str] = None, model_name: Optional[str] = None, model_module: Optional[str] = None, model_class: str = 'make_model', dataset_name: Optional[str] = None, dataset_class: Optional[str] = None, dataset_module: Optional[str] = None)[source]¶ Make a hyperparameter class instance. Required is the hyperparameter dictionary or path to a config file containing the hyperparameter. Furthermore, name of the dataset and model can be provided for checking the information in
hyper_info.- Parameters
hyper_info (str, dict) – Hyperparameter dictionary or path to file.
hyper_category (str) – Category within hyperparameter.
model_name (str, optional) – Name of the model.
model_module (str, optional) – Name of the module of the model. Defaults to None.
model_class (str, optional) – Class name or make function for model. Defaults to ‘make_model’.
dataset_name (str, optional) – Name of the dataset.
dataset_class (str, optional) – Class name of the dataset.
dataset_module (str, optional) – Module name of the dataset.
-
compile(loss=None, optimizer='rmsprop', metrics: Optional[list] = None, weighted_metrics: Optional[list] = None)[source]¶ Generate kwargs for
tf.keras.Model.compilefrom hyperparameter and default parameter.This function should handle deserialization of hyperparameter and, if not specified, fill them from default. Loss, optimizer are overwritten from hyperparameter, if available. Metrics in hyperparameter are added from function arguments. Note that otherwise metrics can not be deserialized, since metrics can include nested lists and a dictionary of model output names.
Warning
When using deserialization with this function, you must not name your model output “class_name”, “module_name” and “config”.
- Parameters
- Returns
Deserialized compile kwargs from hyperparameter.
- Return type
-
property
dataset_class¶
-
property
dataset_module¶
-
property
dataset_name¶
-
fit(epochs: int = 1, validation_freq: int = 1, batch_size: Optional[int] = None, callbacks: Optional[list] = None)[source]¶ Select fit hyperparameter. Additional default values for the training scripts are given as functional kwargs. Functional kwargs are overwritten by hyperparameter.
- Parameters
- Returns
de-serialized fit kwargs from hyperparameter.
- Return type
-
property
hyper_category¶
-
property
model_class¶
-
property
model_module¶
-
property
model_name¶
-
results_file_path()[source]¶ Make output folder for results based on hyperparameter and return path to that folder. The folder is set up as ‘results’/dataset/model_name + post_fix. Where model and dataset name must be set by this class. Postfix can be in hyperparameter setting.
- Returns
File-path or path object to result folder.
- Return type
-
kgcnn.training.schedule module¶
-
class
kgcnn.training.schedule.KerasPolynomialDecaySchedule(dataset_size: int, batch_size: int, epochs: int, lr_start: float = 0.0005, lr_stop: float = 1e-05)[source]¶ Bases:
keras.src.optimizers.schedules.learning_rate_schedule.PolynomialDecayThis schedule inherits from :obj:` ks.optimizers.schedules.PolynomialDecay` .
-
class
kgcnn.training.schedule.LinearWarmupExponentialDecay(learning_rate, warmup_steps, decay_steps, decay_rate, staircase: bool = False)[source]¶ Bases:
keras.src.optimizers.schedules.learning_rate_schedule.LearningRateScheduleThis schedule combines a linear warmup with an exponential decay. Combines :obj:` tf.optimizers.schedules.PolynomialDecay` with an actual increase during warmup and
tf.optimizers.schedules.ExponentialDecayafter.Introduced by DimeNetPP .
The closed-from learning rate schedule for learning rate \(\eta\) for \(s_0\) warmup and decay \(S_\tau\) is given as a function of steps \(s\) below (deduced from documentation of keras modules).
\[\eta (s) = \eta_0 \; \gamma ^ {s / S_\tau} \; [1 - \frac{s_0-1}{s_0} \frac{s_0 - \text{min}(s_0, S)}{s_0}]\]This class has been updated to be compatible with GemNet training.
-
__call__(step)[source]¶ Decay learning rate as a functions of steps.
- Parameters
step – Current step of training.
- Returns
New learning rate.
- Return type
-
__init__(learning_rate, warmup_steps, decay_steps, decay_rate, staircase: bool = False)[source]¶ Initialize class.
- Parameters
learning_rate – Learning rate to use.
warmup_steps – Number of warmup steps.
decay_steps – Number of which to decay the learning rate.
decay_rate – Factor to reduce the learning rate.
staircase (bool) – If True use staircase decay and not (continuous) exponential decay.
-
property
initial_learning_rate¶
-
kgcnn.training.scheduler module¶
-
class
kgcnn.training.scheduler.CosineAnnealingLRScheduler(lr_start: float, epoch_max: int, lr_min: float = 0, verbose: int = 0)[source]¶ Bases:
keras.src.callbacks.learning_rate_scheduler.LearningRateSchedulerCallback for cosine learning rate (LR) schedule. This class inherits from
ks.callbacks.LearningRateSchedulerand appliesschedule_epoch_lr. Proposed by SGDR. The cosine part without restarts for the LR Schedule follows:\[\eta_t (T_{cur}) = \eta_{min} + \frac{1}{2}(\eta_{max} - \eta_{min})\left(1 + \cos\left(\frac{T_{cur}}{T_{max}}\pi\right)\right)\]
-
class
kgcnn.training.scheduler.LinearLearningRateScheduler(learning_rate_start: float = 0.001, learning_rate_stop: float = 1e-05, epo_min: int = 0, epo: int = 500, verbose: int = 0, eps: float = 1e-08)[source]¶ Bases:
keras.src.callbacks.learning_rate_scheduler.LearningRateSchedulerCallback for linear change of the learning rate. This class inherits from
ks.callbacks.LearningRateScheduler. The learning rate \(\eta_0\) is reduced linearly at \(T_0\) epochs up to \(T_{max}\) epochs to reach the learning rate \(\eta_{T_{max}}\):\[\eta (T) = \eta_0 - \frac{\eta_0 - \eta_{T_{max}}}{T_{max}-T_0} (T-T_0) \;\; \text{for} \;\; T>T_0\]-
__init__(learning_rate_start: float = 0.001, learning_rate_stop: float = 1e-05, epo_min: int = 0, epo: int = 500, verbose: int = 0, eps: float = 1e-08)[source]¶ Set the parameters for the learning rate scheduler.
- Parameters
learning_rate_start (float) – Initial learning rate. Default is 1e-3.
learning_rate_stop (float) – End learning rate. Default is 1e-5.
epo_min (int) – Minimum number of epochs to keep the learning-rate constant before decrease. Default is 0.
epo (int) – Total number of epochs. Default is 500.
eps (float) – Minimum learning rate. Default is 1e-08.
verbose (int) – Verbosity. Default is 0.
-
-
class
kgcnn.training.scheduler.LinearWarmUpScheduler(schedule=None, verbose: int = 0, steps_per_epoch: Optional[int] = None, lr_start: Optional[float] = None, epo_warmup: int = 0)[source]¶ Bases:
keras.src.callbacks.learning_rate_scheduler.LearningRateSchedulerWarmup scheduler that increases the learning rate over the first steps or epochs. Inherits from
ks.callbacks.LearningRateScheduler. Note that the parameterschedulein the constructor can be None which sets the wam-up inlinear_warmup_schedule_epoch_lronly. Another learning rate schedule can be provided that makes use oflinear_warmup_schedule_epoch_lrin subclasses.Note
To increase the learning rate within each epoch, you must provide
steps_per_epoch.-
__init__(schedule=None, verbose: int = 0, steps_per_epoch: Optional[int] = None, lr_start: Optional[float] = None, epo_warmup: int = 0)[source]¶ Initialize class.
- Parameters
schedule (Callable) – Learning rate schedule. Default is None.
verbose (int) – Verbosity. Default is 0.
steps_per_epoch (int) – Steps per epoch. Required for sub-epoch increase. Default is None.
lr_start (float) – Learning rate to reach in linear ramp to start learning. Default is None.
epo_warmup (int) – Number of warmup epochs. Default is None.
-
-
class
kgcnn.training.scheduler.LinearWarmupExponentialLRScheduler(lr_start: float, gamma: float, epo_warmup: int = 10, lr_min: float = 0.0, verbose: int = 0, steps_per_epoch: Optional[int] = None)[source]¶ Bases:
kgcnn.training.scheduler.LinearWarmUpSchedulerCallback for exponential learning rate schedule with warmup. This class inherits from
LinearWarmUpScheduler, which inherits fromks.callbacks.LearningRateScheduler. The learning rate \(\eta\) is reduced or increased (usually \(\gamma < 1\)) after warmup epochs \(T_0\) as:\[\eta (T) = \eta_0 \; \gamma^{T-T_0}\]-
__init__(lr_start: float, gamma: float, epo_warmup: int = 10, lr_min: float = 0.0, verbose: int = 0, steps_per_epoch: Optional[int] = None)[source]¶ Set the parameters for the learning rate scheduler.
- Parameters
lr_start (float) – Learning rate at the start of the exponential decay.
gamma (float) – Multiplicative factor of learning rate decay.
epo_warmup (int) – Number of warmup epochs. Default is 10.
lr_min (float) – Minimum learning rate allowed during the decay and start. Default is 0.0.
verbose (int) – Verbosity. Default is 0.
steps_per_epoch (int) – Number of steps per epoch. Required for warmup to linearly increase between epochs. Default is None.
-
-
class
kgcnn.training.scheduler.LinearWarmupExponentialLearningRateScheduler(lr_start: float, decay_lifetime: float, epo_warmup: int = 10, lr_min: float = 0.0, verbose: int = 0, steps_per_epoch: Optional[int] = None)[source]¶ Bases:
kgcnn.training.scheduler.LinearWarmUpSchedulerCallback for exponential learning rate schedule with warmup. This class inherits from
LinearWarmUpScheduler, which inherits fromks.callbacks.LearningRateScheduler. The learning rate \(\eta\) is reduced after warmup epochs \(T_0\) with lifetime \(\tau\) as:\[\eta (T) = \eta_0 \; e^{-\frac{T-T_0}{\tau}}\]-
__init__(lr_start: float, decay_lifetime: float, epo_warmup: int = 10, lr_min: float = 0.0, verbose: int = 0, steps_per_epoch: Optional[int] = None)[source]¶ Set the parameters for the learning rate scheduler.
- Parameters
lr_start (float) – Learning rate at the start of the exp. decay.
decay_lifetime (float) – Tau or lifetime parameter in the exponential in epochs.
epo_warmup (int) – Number of warmup epochs. Default is 10.
lr_min (float) – Minimum learning rate allowed during the decay. Default is 0.0.
verbose (int) – Verbosity. Default is 0.
steps_per_epoch (int) – Number of steps per epoch. Required for warmup to linearly increase between epochs. Default is None.
-
-
class
kgcnn.training.scheduler.LinearWarmupLinearLearningRateScheduler(learning_rate_start: float = 0.001, learning_rate_stop: float = 1e-05, epo_warmup: int = 0, epo: int = 500, verbose: int = 0, eps: float = 1e-08, steps_per_epoch: Optional[int] = None, lr_start: Optional[int] = None)[source]¶ Bases:
kgcnn.training.scheduler.LinearWarmUpSchedulerCallback for linear change of the learning rate with warmup. This class inherits from
LinearWarmUpScheduler, which inherits fromks.callbacks.LearningRateScheduler. The learning rate is increased in a warmup phase of \(T_0\) epochs up to \(\eta_0\). The learning rate \(\eta_0\) is reduced linearly at \(T_0\) epochs up to \(T_{max}\) epochs to reach the learning rate \(\eta_{T_{max}}\):\[\eta (T) = \eta_0 - \frac{\eta_0 - \eta_{T_{max}}}{T_{max}-T_0} (T-T_0) \;\; \text{for} \;\; T>T_0\]-
__init__(learning_rate_start: float = 0.001, learning_rate_stop: float = 1e-05, epo_warmup: int = 0, epo: int = 500, verbose: int = 0, eps: float = 1e-08, steps_per_epoch: Optional[int] = None, lr_start: Optional[int] = None)[source]¶ Set the parameters for the learning rate scheduler.
- Parameters
learning_rate_start (float) – Initial learning rate. Default is 1e-3.
learning_rate_stop (float) – End learning rate. Default is 1e-5.
epo (int) – Total number of epochs. Default is 500.
eps (float) – Minimum learning rate. Default is 1e-08.
verbose (int) – Verbosity. Default is 0.
steps_per_epoch (int) – Number of steps per epoch. Required for warmup to linearly increase between epochs. Default is None.
lr_start (int) – Ignored, set to learning_rate_start.
-
-
class
kgcnn.training.scheduler.PolynomialDecayScheduler(initial_learning_rate, decay_epochs, end_learning_rate=0.0001, power=1.0, cycle=False, verbose: int = 0, eps: float = 1e-08)[source]¶ Bases:
keras.src.callbacks.learning_rate_scheduler.LearningRateSchedulerCallback for polynomial decay of the learning rate. This class inherits from
ks.callbacks.LearningRateScheduler.Adapts
keras.optimizers.schedules.PolynomialDecayto a scheduler with a function of epochs.-
__init__(initial_learning_rate, decay_epochs, end_learning_rate=0.0001, power=1.0, cycle=False, verbose: int = 0, eps: float = 1e-08)[source]¶ Set the parameters for the learning rate scheduler.
- Parameters
initial_learning_rate (float) – The initial learning rate
decay_epochs (float) – Must be positive. See the decay computation above.
end_learning_rate (float) – The minimal end learning rate.
power (float) – The power of the polynomial. Defaults to 1.0 .
cycle (bool) – A boolean, whether it should cycle beyond decay_steps.
eps (float) – Minimum learning rate. Default is 1e-08.
verbose (int) – Verbosity. Default is 0.
-