TensorFlow Lite: Deploying on Mobile

A model that runs beautifully on a desktop GPU is not automatically usable on a phone. Mobile devices have limited memory, limited compute, and battery constraints that desktop training never has to consider. TensorFlow Lite (TFLite) is the toolkit built specifically to shrink and run TensorFlow models efficiently on mobile and embedded hardware.

Introduction to TensorFlow Lite

TensorFlow Lite is a lightweight runtime designed for on-device inference. Instead of sending user data to a server and waiting for a response, a TFLite model runs directly on the device, which gives you three big advantages:

•        Lower latency: predictions happen instantly, with no network round-trip.

•        Offline capability: the app works even with no internet connection.

•        Better privacy: sensitive data, like camera frames or voice recordings, never has to leave the device.

The trade-off is that mobile hardware is far more constrained than a training server, so TFLite focuses heavily on making models smaller and faster, sometimes at a small cost in accuracy.

Converting Models to TFLite Format

The TFLite Converter takes a trained Keras or SavedModel and converts it into a .tflite FlatBuffer file, a compact binary format optimised for fast loading and low memory overhead.

import tensorflow as tf

# Starting from a trained Keras model

model = tf.keras.models.load_model('my_model.h5')

converter = tf.lite.TFLiteConverter.from_keras_model(model)

tflite_model = converter.convert()

with open('model.tflite', 'wb') as f:

f.write(tflite_model)

You can also convert directly from a SavedModel directory, which is the more common path in production pipelines:

converter = tf.lite.TFLiteConverter.from_saved_model('saved_model_dir')

tflite_model = converter.convert()

Model Quantization

Quantization reduces the precision used to store and compute weights, typically from 32-bit floats down to 8-bit integers. This shrinks the model size by roughly 4x and speeds up inference, since integer arithmetic is cheaper than floating-point arithmetic on most mobile CPUs.

converter = tf.lite.TFLiteConverter.from_keras_model(model)

converter.optimizations = [tf.lite.Optimize.DEFAULT]

# Full integer quantization needs a representative dataset

def representative_dataset():

for sample in x_train[:500]:

yield [sample.reshape(1, 784).astype('float32')]

converter.representative_dataset = representative_dataset

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]

tflite_quant_model = converter.convert()

The representative dataset is used to calibrate the quantization ranges, TFLite observes the typical range of activation values on real samples so it can choose integer scales that minimise information loss.

Running Inference on Android

On Android, the TFLite Interpreter loads the .tflite file and runs predictions using Kotlin or Java. The model file is typically placed in the app's assets folder.

// Kotlin

val model = FileUtil.loadMappedFile(context, "model.tflite")

val interpreter = Interpreter(model)

val input = arrayOf(floatArrayOf(/* 784 normalised pixel values */))

val output = Array(1) { FloatArray(10) }

interpreter.run(input, output)

val predictedClass = output[0].indices.maxByOrNull { output[0][it] }

println("Predicted digit: $predictedClass")

For common tasks like image classification or object detection, the TFLite Task Library wraps this boilerplate into a few lines, handling input pre-processing and output parsing automatically.

Running Inference on iOS

On iOS, the same .tflite file is loaded through the TensorFlowLiteSwift pod, with a near-identical API shape to the Android interpreter.

// Swift

import TensorFlowLite

let modelPath = Bundle.main.path(forResource: "model", ofType: "tflite")!

let interpreter = try Interpreter(modelPath: modelPath)

try interpreter.allocateTensors()

let inputData = Data(copyingBufferOf: inputPixels)

try interpreter.copy(inputData, toInputAt: 0)

try interpreter.invoke()

let outputTensor = try interpreter.output(at: 0)

let probabilities = outputTensor.data.toArray(type: Float32.self)

Quick Tip

Both platforms support hardware delegates, GPU delegate, NNAPI on Android, and Core ML delegate on iOS, that offload computation to specialised hardware. Enabling a delegate can speed up inference significantly with no changes to the model itself.