Working of TensorFlow

Before writing your first model, it is worth spending a few minutes understanding the core ideas that everything else is built on. TensorFlow's architecture rests on two concepts: tensors and computational graphs, and its execution model in TF 2.x gives you the best of both worlds.

Tensors Explained Simply

A tensor is a generalisation of numbers, vectors, and matrices to any number of dimensions. The name sounds exotic, but you work with tensors in everyday life without calling them that:

• Scalar (rank-0 tensor): a single number — e.g. the temperature outside right now.
• Vector (rank-1 tensor): a list of numbers, e.g. the pixel brightness values along one row of an image.
• Matrix (rank-2 tensor): a grid of numbers, e.g. a grayscale image where each value is one pixel's brightness.
• 3-D tensor: a stack of matrices, e.g. a colour image with shape (height, width, 3) where 3 is the RGB channel count.
• 4-D tensor: a batch of colour images with shape (batch_size, height, width, 3) — this is what most image models receive during training.

TensorFlow represents all data inputs, labels, model weights, intermediate activations, and gradients as tensors. According to the official TensorFlow documentation, TensorFlow operates on multidimensional arrays or tensors represented as tf. Tensor objects, similar to NumPy arrays but able to run on GPUs and participate in automatic differentiation.

import tensorflow as tf

# Rank-0: scalar

scalar = tf.constant(42.0)

print(scalar.shape)   # ()

print(scalar.dtype)   # float32

# Rank-1: vector

vector = tf.constant([1.0, 2.0, 3.0, 4.0])

print(vector.shape)   # (4,)

# Rank-2: matrix

matrix = tf.constant([[1, 2, 3],

[4, 5, 6]])

print(matrix.shape)   # (2, 3)

# TensorFlow offers a rich operator library

print(tf.math.add(1, 2))          # tf.Tensor(3, ...)

print(tf.math.square(5))          # tf.Tensor(25, ...)

print(tf.math.reduce_sum([1,2,3]))   # tf.Tensor(6, ...)

print(tf.matmul(matrix, tf.transpose(matrix)))  # 2x2 result

Did You Know?

TensorFlow tensors are immutable by default — like Python strings, you cannot change a value in place. For mutable state (such as model weights that update during training), TensorFlow provides tf.Variable, which supports in-place updates via assign() and assign_add().

Computational Graphs in TensorFlow

A computational graph is a directed graph where each node represents a mathematical operation (tf.Operation) and each edge represents the tensor flowing between operations (tf.Tensor). According to the official TensorFlow guide, since these graphs are data structures, they can be saved, run, and restored all without the original Python code.

Why does the graph matter? Because once TensorFlow has the full picture of all operations in a function, it can hand them to Grappler, TensorFlow's built-in graph optimisation engine, which performs:

• Constant folding: evaluates expressions involving only constants at compile time rather than runtime.
• Common subexpression elimination: avoids computing the same value twice.
• Operation fusion: merges multiple small operations into a single efficient kernel, reducing memory bandwidth.
• Automatic parallelisation: identifies independent sub-computations and executes them concurrently.

Eager Execution vs Graph Execution

TensorFlow 1.x required you to define the full graph upfront, then open a Session to run it, a design that was powerful but notoriously difficult to debug. A Medium article from November 2025 described the frustration clearly: 'After executing the code, there was occasionally no output. The challenge was not a coding error; it was how TensorFlow 1.x was designed to operate.'

TensorFlow 2.x changed that by enabling eager execution by default. Operations execute immediately as Python calls them, just like NumPy, no sessions, no graphs to compile manually. The official TensorFlow documentation states: 'As of TensorFlow 2, eager execution is turned on by default. Eager execution enables a more interactive frontend to TensorFlow.'

import tensorflow as tf

# Eager (default

print(f'Eager enabled: {tf.executing_eagerly()}')  # True

x = tf.constant([[1.0, 2.0], [3.0, 4.0]])

y = tf.constant([[5.0, 6.0], [7.0, 8.0]])

z = tf.matmul(x, y)   # executes immediately

print(z.numpy())    # [[19. 22.] [43. 50.]]

Graph (via decorator)

@tf.function

def square_and_sum(a, b):

return tf.reduce_sum(a**2 + b**2)

result = square_and_sum(x, y)

print(result)  # tf.Tensor(190.0, ...)

# Toggle eager off to verify graph mode (useful during debugging)

# tf.config.run_functions_eagerly(True) runs @tf.function eagerly

Recommended Practice

Use eager execution during development and debugging, the immediate feedback is invaluable. Once your model is verified and correct, add @tf.function to performance-critical sections (the training step, the inference function) and let TensorFlow compile and optimise them for speed.