MLOps with TensorFlow

Everything covered so far- custom training, distributed scaling, pipelines, mobile deployment, serving, optimisation- eventually needs to operate together as a continuous, monitored system. MLOps is the discipline that ties these pieces into a sustainable process rather than a one-off project.

Introduction to MLOps

MLOps borrows heavily from DevOps but adds concerns unique to machine learning: data changes over time, models degrade silently, and the same code can produce different results depending on the data it was trained on. A mature MLOps practice covers:

• Versioning everything: not just code, but datasets, schemas, and trained models, so any result can be reproduced.

• Automated, repeatable training: pipelines (like TFX, covered in Chapter 3) instead of manual notebook runs.

• Continuous monitoring: tracking how a deployed model performs over time, not just at the moment it is launched.

• Fast, safe rollback: the ability to revert to a previous model version quickly if something goes wrong.

None of this is optional once a model affects real users or real decisions; without it, small data shifts quietly become large, undetected accuracy losses.

Experiment Tracking

Before a model ever reaches production, it usually goes through dozens or hundreds of experiments, different architectures, hyperparameters, and datasets. Experiment tracking tools record every run's configuration, metrics, and artifacts so you can compare them later and reproduce the best one.

import mlflow

import tensorflow as tf

mlflow.set_experiment('digit_classifier')

with mlflow.start_run():

mlflow.log_param('learning_rate', 0.001)

mlflow.log_param('batch_size', 64)

model = tf.keras.Sequential([

tf.keras.layers.Dense(256, activation='relu', input_shape=(784,)),

tf.keras.layers.Dense(10, activation='softmax'),

])

model.compile(optimizer='adam',

loss='sparse_categorical_crossentropy',

metrics=['accuracy'])

history = model.fit(x_train, y_train, epochs=10, validation_split=0.1)

mlflow.log_metric('final_val_accuracy', history.history['val_accuracy'][-1])

mlflow.tensorflow.log_model(model, 'model')

TensorBoard plays a complementary role for visualising training curves, weight histograms, and computation graphs in real time during a single run, while tools like MLflow and Weights & Biases focus on comparing many runs against each other.

Continuous Training Pipelines

A continuous training (CT) pipeline automatically retrains a model when new data arrives or performance drops, rather than waiting for a person to notice and manually kick off training. This typically combines a TFX pipeline (Chapter 3) with an orchestrator schedule and a trigger condition.

# Simplified Airflow DAG trigger for a TFX pipeline

from airflow import DAG

from airflow.operators.python import PythonOperator

from datetime import datetime

def run_tfx_pipeline():

from tfx.orchestration.local.local_dag_runner import LocalDagRunner

LocalDagRunner().run(pipeline)

with DAG('weekly_retrain', schedule_interval='@weekly',

start_date=datetime(2026, 1, 1)) as dag:

retrain_task = PythonOperator(

task_id='retrain_model',

python_callable=run_tfx_pipeline,

)

Model Monitoring

A model's accuracy on its original test set tells you nothing about how it performs six months later. Two forms of drift quietly erode model quality in production:

• Data drift: the statistical distribution of incoming features changes over time, e.g. user demographics shift, or a sensor is recalibrated.

• Concept drift: the relationship between inputs and the correct output changes, e.g. customer preferences evolve, making old patterns less predictive.

Monitoring systems compare live, incoming feature distributions against the training distribution (often using the same TFDV tooling from Chapter 3) and track proxy signals like prediction confidence, latency, and, where ground truth eventually becomes available, live accuracy.

import tensorflow_data_validation as tfdv

# Compare live serving data against the original training schema

serving_stats = tfdv.generate_statistics_from_csv(data_location='live_traffic.csv')

drift_anomalies = tfdv.validate_statistics(

statistics=serving_stats,

schema=training_schema,

previous_statistics=training_stats,

)

tfdv.display_anomalies(drift_anomalies)

Did You Know?

Concept drift is often harder to detect than data drift because the input data itself can look perfectly normal; the meaning behind it is what has changed. This is why monitoring live model accuracy, wherever ground-truth labels eventually become available, remains essential even when input distributions look stable.

Managing ML Lifecycle

A model registry ties experiment tracking, training pipelines, and serving together by acting as the single source of truth for which model version is currently approved, staged, or in production. Most registries enforce a clear lifecycle:

import mlflow

client = mlflow.tracking.MlflowClient()

# Promote a specific model version to Production

client.transition_model_version_stage(

name='digit_classifier',

version=4,

stage='Production',

archive_existing_versions=True,   # auto-archive the old Production model

)

With versioned data, tracked experiments, automated pipelines, continuous monitoring, and a clear registry lifecycle in place, you have the complete picture: a model is no longer a one-time deliverable but a living system that can be safely updated, rolled back, and improved for as long as it remains in production.