TensorFlow Callbacks and Logging

A callback is a function that Keras calls at specific stages of training after every batch, after every epoch, at the start and end of training. As described in the official TensorFlow callback writing guide: 'A callback is a powerful tool to customize the behavior of a Keras model during training, evaluation, or inference.' They let you monitor, control, and automate what would otherwise require manual intervention.

Introduction to Callbacks

You pass callbacks as a list to model.fit(). At runtime, Keras invokes the appropriate method (on_epoch_end, on_batch_end, etc.) automatically. You can combine as many callbacks as you need:

callbacks = [

tf.keras.callbacks.ModelCheckpoint(...),

tf.keras.callbacks.EarlyStopping(...),

tf.keras.callbacks.ReduceLROnPlateau(...),

tf.keras.callbacks.TensorBoard(...),

]

model.fit(x_train, y_train, epochs=100, callbacks=callbacks)

ModelCheckpoint

ModelCheckpoint saves the model automatically during training. Setting save_best_only=True ensures only the best version is kept no need to monitor training and intervene manually:

checkpoint = tf.keras.callbacks.ModelCheckpoint(

filepath='best_model.keras',

monitor='val_accuracy', # watch validation accuracy

save_best_only=True,    # only save when val_accuracy improves

save_weights_only=False,   # save full model (architecture + weights)

verbose=1

)

# When training completes, best_model.Keras contains the best checkpoint

# Load it after training to ensure you have the peak-performance version

best_model = tf.keras.models.load_model('best_model.keras')

EarlyStopping

EarlyStopping monitors a validation metric and halts training when it stops improving. This prevents overfitting and wasted compute. The patience parameter controls how many consecutive non-improving epochs to tolerate before stopping:

early_stop = tf.keras.callbacks.EarlyStopping(

monitor='val_loss',     # watch validation loss

patience=7,             # wait 7 epochs before stopping

restore_best_weights=True, # revert to best weights when stopped

min_delta=0.001,        # minimum improvement to count as progress

verbose=1

)

# Training will stop early if val_loss doesn't improve for 7 epochs

# restore_best_weights=True is critical — without it, you get the last

# (possibly worse) model, not the best one

Common Mistake

Always set restore_best_weights=True in EarlyStopping. Without it, when training stops, you get the weights from the final epoch, which may be worse than the best checkpoint, defeating the purpose of early stopping.

ReduceLROnPlateau

When training stalls — loss stops improving but has not yet converged — reducing the learning rate often breaks the plateau and allows further optimisation. ReduceLROnPlateau does this automatically:

reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(

monitor='val_loss',

factor=0.2,      # multiply LR by 0.2 when triggered

patience=3,      # wait 3 epochs before reducing

min_lr=1e-6,     # never go below this learning rate

verbose=1

)

# Example: LR starts at 0.001

# After 3 non-improving epochs: LR → 0.0002

# After 3 more: LR → 0.00004 — and so on down to min_lr

Custom Callbacks

When built-in callbacks are not enough, you can create your own by subclassing tf.keras.callbacks.Callback and overriding the methods you need. The official TensorFlow guide provides a complete list of available hook methods:

class LossLogger(tf.keras.callbacks.Callback):

""Prints loss and learning rate at the end of each epoch."""

def on_train_begin(self, logs=None):

print('Training started!')

def on_epoch_end(self, epoch, logs=None):

lr = float(self.model.optimizer.learning_rate)

loss = logs.get('loss')

val_loss = logs.get('val_loss')

print(f'Epoch {epoch+1:3d}: loss={loss:.4f}, '

f'val_loss={val_loss:.4f}, lr={lr:.6f}')

def on_train_end(self, logs=None):

print('Training complete!')

model.fit(x_train, y_train, epochs=10,

callbacks=[LossLogger()],

validation_split=0.1)

TensorBoard Overview

TensorBoard is TensorFlow's built-in visualisation toolkit. It runs as a local web server and provides interactive dashboards for monitoring training. You can track loss and accuracy curves in real time, inspect model architecture graphs, view weight histograms, and compare multiple training runs side by side, all without writing any additional code.

Visualizing Training with TensorBoard

import datetime, tensorflow as tf

# Each run gets a timestamped subdirectory — keeps runs separate

log_dir = 'logs/fit/' + datetime.datetime.now().strftime('%Y%m%d-%H%M%S')

tensorboard_cb = tf.keras.callbacks.TensorBoard(

log_dir=log_dir,

histogram_freq=1,   # log weight histograms every epoch

write_graph=True,   # log the model computation graph

write_images=False, # optionally visualise layer weights as images

update_freq='epoch' # log after every epoch (or 'batch' for finer detail)

)

model.fit(x_train, y_train, epochs=20,

validation_split=0.1,

callbacks=[tensorboard_cb])

# Launch TensorBoard in the terminal:

# tensorboard --logdir logs/fit

# Then open http://localhost:6006 in your browser