Transfer Learning with TensorFlow

Training a deep neural network from scratch requires millions of labelled examples and days or weeks of compute time. Transfer learning short-circuits this entirely; you take a model already trained on a massive dataset and adapt it to your task. It is one of the most practically important techniques in real-world deep learning.

What is Transfer Learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. As described in the Keras documentation: 'For instance, features from a model that has learned to identify raccoons may be useful to kick-start a model meant to identify tanukis.'

The central insight is that the early layers of a deep CNN learn universal visual features, edges, textures, gradients, and colour blobs that are useful for almost any image task. Only the final classification layers need to be tailored to the specific classes in your dataset.t

Why Use Transfer Learning?

• Data efficiency: you can achieve excellent results with as few as a few hundred labelled images per class, because the backbone already knows how to see.
• Faster training: you freeze the pretrained backbone and only train a small classification head, reducing training time from days to minutes.
• Better performance: on small datasets, a pretrained model almost always outperforms a model trained from scratch, even after extensive tuning.
• Reduced compute cost: fine-tuning requires far fewer GPU-hours than pretraining

Using Pre-trained Models

TensorFlow's tf.keras.The applications module provides a collection of state-of-the-art image classification models with pretrained ImageNet weights. The include_top=False argument removes the final classification layers, leaving only the feature extraction backbone:

import tensorflow as tf

IMG_SIZE = (160, 160)

IMG_SHAPE = IMG_SIZE + (3,)   # (160, 160, 3)

# Download MobileNetV2 without the classification head

base_model = tf.keras.applications.MobileNetV2(

input_shape=IMG_SHAPE,

include_top=False,     # exclude the final Dense classifier

weights='imagenet'     # use pretrained ImageNet weights

)

print(base_model.output_shape)  # (None, 5, 5, 1280)

MobileNet

MobileNetV2, developed at Google, is designed for mobile and edge applications. It uses depthwise separable convolutions, a mathematical trick that achieves similar accuracy to standard convolutions but with roughly 8–9x fewer operations. The smallest version processes images in milliseconds on a smartphone CPU. It is the default choice when you need on-device inference.

ResNet

ResNet (Residual Network), introduced by Microsoft Research in 2015, solved a fundamental problem: very deep networks (50+ layers) were paradoxically performing worse than shallower ones due to degradation. ResNet's solution was the residual connection, a shortcut that adds the input of a block directly to its output. This allows gradients to flow unchanged through many layers, enabling networks of 50, 101, or even 152 layers to train successfully. Use ResNet50V2 when accuracy matters more than inference speed.

Feature Extraction

In feature extraction mode, you freeze the entire pretrained backbone and train only a new classification head. The backbone acts as a fixed feature extractor; it converts your images into rich feature vectors that your new Dense layers classify:

# Freeze the backbone, its weights will not update during training

base_model.trainable = False

print(f'Trainable layers: {len(base_model.trainable_variables)}')  # 0

# Add a classification head on top

inputs  = tf.keras.Input(shape=IMG_SHAPE)

# preprocess_input scales pixels for MobileNetV2 (required: -1 to 1 range)

x = tf.keras.applications.mobilenet_v2.preprocess_input(inputs)

x = base_model(x, training=False)   # training=False: keep BN in inference mode

x = tf.keras.layers.GlobalAveragePooling2D()(x)   # (None, 1280)

x = tf.keras.layers.Dropout(0.2)(x)

outputs = tf.keras.layers.Dense(2, activation='softmax')(x)  # 2 classes

model = tf.keras.Model(inputs, outputs)

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),

loss='sparse_categorical_crossentropy',

metrics=['accuracy'])

Fine-Tuning a Pre-trained Model

After training the classification head, you can unlock some of the later backbone layers and train them together with the head at a very low learning rate. This allows the backbone to adapt its high-level features to your specific domain. The official TensorFlow transfer learning guide warns: always fine-tune at a learning rate at least 10x lower than the initial training, or you risk destroying the pretrained weights:

# After training the head for ~10 epochs, unfreeze the top layers

base_model.trainable = True

# Only fine-tune the last 30 layers of MobileNetV2

# Earlier layers learn universal features — no need to update them

fine_tune_at = len(base_model.layers) - 30

for layer in base_model.layers[:fine_tune_at]:

layer.trainable = False

# Recompile with a MUCH lower learning rate

model.compile(

optimizer=tf.keras.optimizers.Adam(learning_rate=1e-5),  # 100x lower

loss='sparse_categorical_crossentropy',

metrics=['accuracy']

)

model.fit(train_ds, epochs=10, validation_data=val_ds)

Transfer Learning Workflow

The official Keras transfer learning guide outlines a clear, repeatable workflow that applies to virtually any image task: