Building Your First Neural Network

You have the theory. Now, let us build a complete neural network end-to-end from raw data to trained model to live predictions. We use MNIST, the canonical benchmark, so you can compare your results with millions of other practitioners worldwide.

Preparing the Dataset (MNIST)

MNIST ships inside TensorFlow. One call loads 60,000 training images and 10,000 test images, each a 28x28 grayscale picture of a handwritten digit (0–9).

import tensorflow as tf

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

print(x_train.shape)   # (60000, 28, 28)

print(y_train[:5])     # [5 0 4 1 9]

Data Preprocessing and Normalization

Raw pixel values run from 0 to 255. Large input values destabilise training because they produce large activations, which in turn produce large gradients, causing unstable weight updates. Dividing by 255 scales everything to 0–1, which aligns with the typical range of initial weights.

# Normalise to 0–1 and flatten 28x28 → 784

x_train = x_train.reshape(-1, 784).astype('float32') / 255.0

x_test  = x_test.reshape(-1, 784).astype('float32')  / 255.0

print(x_train.shape)   # (60000, 784)

print(x_train.min(), x_train.max())  # 0.0  1.0

Building the Model with tf.keras

The Sequential API lets you define a network by stacking layers in order. Each layer takes the output of the previous one as input:
model = tf.keras.Sequential([

tf.keras.layers.InputLayer(input_shape=(784,)),

tf.keras.layers.Dense(256, activation='relu'),

tf.keras.layers.Dropout(0.3),

tf.keras.layers.Dense(128, activation='relu'),

tf.keras.layers.Dropout(0.2),

tf.keras.layers.Dense(10,  activation='softmax'),

])

model.summary()   # shows layer shapes and ~235,146 total parameters

Compiling the Model

Compiling binds the optimizer, loss function, and evaluation metrics to the model. You must compile before training:

model.compile(

optimizer='adam',

loss='sparse_categorical_crossentropy',  # labels are integers

metrics=['accuracy'])

Training the Model

history = model.fit(

x_train, y_train,

epochs=15,

batch_size=64,

validation_split=0.1,   # 10% of training data for validation

verbose=1
**)**

2.6 Evaluating Model Performance

model.evaluate() runs the model on data it has never seen during training. This gives you a fair measure of generalization:

test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)

print(f'Test Loss:  {test_loss:.4f}')

print(f'Test Accuracy: {test_acc:.4f}')   # typically ~0.9800+

Making Predictions

After training, use model.predict() to get the probability distribution over classes for new inputs. The predicted class is the index with the highest probability:

Get probabilities for 5 test images

probs = model.predict(x_test[:5])   # shape: (5, 10)

preds = probs.argmax(axis=1)     # pick the class with the highest prob

print('Predicted:', preds)

print('Actual:   ', y_test[:5])

# Confidence of top prediction

for i, (p, a) in enumerate(zip(preds, y_test[:5])):

conf = probs[i][p] * 100
print(f'Image {i}: predicted={p}, actual={a}, confidence={conf:.1f}%')

Improving Model Accuracy

• Add more neurons or layers: a deeper or wider network has more capacity to learn complex patterns. Start small and scale up when needed.
• Increase Dropout: if validation accuracy is much lower than training accuracy, the model is overfitting. Increase dropout rates (0.3–0.5) to regularise.
• Try batch normalisation: adding tf.keras.layers.BatchNormalization() after Dense layers stabilises training and often allows higher learning rates.
• Tune the learning rate: Adam's default of 0.001 is good, but sometimes 0.0001 converges more smoothly. Use ReduceLROnPlateau callback (covered in Section 7) to adapt automatically.
• Train for more epochs with EarlyStopping: set a high epoch count and let EarlyStopping halt when validation loss stops improving.

Expected Result

After 15 epochs with the architecture above, you should see test accuracy above 98%. If you are below 96%, check that pixel values are normalised to 0–1 and that you are using sparse_categorical_crossentropy (not categorical_crossentropy).