Deep Learning Fundamentals
Deep learning models process information through interconnected computational units called artificial neurons. These neurons are organised into layers that transform input features into predictions.
For example, an image classification model may learn simple visual patterns in early layers and combine them into more complex representations in later layers.
Deep learning is a subset of machine learning that uses artificial neural networks with multiple computational layers to learn hierarchical representations of data.
Its applications include fraud detection, medical image analysis, language translation, and speech recognition.
What Is the Difference Between Machine Learning and Deep Learning?
Both approaches learn patterns from data, but they differ in model structure and feature representation.
| Feature | Traditional Machine Learning | Deep Learning |
|---|---|---|
| Main Algorithms | Linear models, trees, SVMs | Multilayer neural networks |
| Feature Engineering | Often requires more manual design | Can learn representations automatically |
| Data Requirements | Vary by algorithm | Often substantial |
| Computation | Often less demanding | Frequently GPU-intensive |
| Interpretability | Some models are readily interpretable | Often more difficult to interpret |
| Common Applications | Tabular prediction | Images, language, audio |
Deep learning is not automatically better than traditional machine learning.
For many structured business datasets, tree-based models and linear methods remain strong alternatives.
What Is an Artificial Neural Network?
An Artificial Neural Network (ANN) is a computational model containing layers of connected artificial neurons.
A basic feedforward network contains three types of layers.
Conceptual structure of a feedforward neural network. Connections carry learned weights between layers.
Input Layer
The input layer receives features.
For example, a customer churn model may use monthly spending, account age, and purchase frequency.
Hidden Layers
Hidden layers apply learned mathematical transformations to the inputs.
Multiple hidden layers allow the network to represent complex relationships.
Output Layer
The output layer generates the final result.
A regression network may produce a numerical prediction, while a classification network may produce class probabilities.
How Does an Artificial Neuron Work?
An artificial neuron computes a weighted sum of inputs, adds a bias, and applies an activation function.
\[ z=\sum_{i=1}^{n}w_ix_i+b \]
\[ a=f(z) \]
Here:
- \(x_i\) represents input features.
- \(w_i\) represents learned weights.
- \(b\) represents the bias.
- \(z\) represents the weighted sum.
- \(f\) represents the activation function.
- \(a\) represents the neuron's output.
Numerical Example
Suppose:
\[ x_1=2,\quad x_2=3 \]
\[ w_1=0.5,\quad w_2=0.8,\quad b=0.2 \]
Then:
\[ z=(2\times0.5)+(3\times0.8)+0.2 \]
\[ z=3.6 \]
The output depends on the activation function applied to 3.6.
What Are Activation Functions?
Activation functions transform neuron outputs and help neural networks represent nonlinear patterns.
Without nonlinear activations, consecutive linear layers would be equivalent to a single linear transformation.
ReLU
Rectified Linear Unit (ReLU) is defined as:
\[ ReLU(x)=\max(0,x) \]
It returns zero for negative inputs and retains positive values.
ReLU is widely used in hidden layers because it is computationally simple.
Sigmoid
\[ \sigma(x)=\frac{1}{1+e^{-x}} \]
Sigmoid maps numerical values between 0 and 1.
It is commonly used in binary classification output layers.
Tanh
\[ \tanh(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}} \]
Tanh maps values between -1 and 1.
It appears in certain recurrent neural network components.
Softmax
Softmax converts a vector of scores into values that sum to 1.
\[ Softmax(z_i)=\frac{e^{z_i}}{\sum_j e^{z_j}} \]
It is commonly used for mutually exclusive multiclass classification.
Activation Functions Compared
| Function | Output Range | Common Use |
|---|---|---|
| ReLU | 0 to infinity | Hidden layers |
| Sigmoid | 0 to 1 | Binary classification output |
| Tanh | -1 to 1 | Certain recurrent architectures |
| Softmax | Class probabilities summing to 1 | Multiclass output |
| Linear | Unrestricted | Regression output |
The output activation should match the prediction task and selected loss function.
How Do Neural Networks Learn?
Neural networks learn by adjusting their weights and biases to reduce a loss function.
A simplified process includes:
- Input features enter the network.
- Layers calculate predictions.
- A loss function measures prediction error.
- Gradients are computed using backpropagation.
- An optimiser updates the model parameters.
The process repeats over batches and epochs.
An epoch represents one complete pass through the training dataset.
What Are the Advantages and Limitations of Deep Learning?
Deep learning can learn complex nonlinear representations and perform well on high-dimensional data such as images and text.
However, it may require considerable computational resources, careful tuning, and representative training data.
Its predictions can also be difficult to explain, and models may perform poorly when new data differs substantially from training data.
Common Mistakes in Neural Network Fundamentals
- Assuming more layers always improve performance.
- Confusing artificial neurons with biological neurons.
- Selecting unsuitable output activation functions.
- Ignoring feature scaling and data quality.
- Assuming deep learning removes the need for model evaluation.
Data Science Tutorial for Beginners
Vaishali Ardhana








