How Neural Networks Work

A neural network is made up of small units called neurons, organized into layers, that pass information forward to produce a prediction. Each connection between neurons carries a weight, and each neuron has a bias, both of which get adjusted as the network learns.

Activation functions are applied along the way to help the network capture patterns that are not simply straight lines. Together, these parts let a neural network turn raw input into a useful output.

What is a Neuron?

A neuron in a neural network is the basic unit that takes in some numbers, does a small calculation, and passes the result forward. A neural network consists of connected units or nodes called artificial neurons, which loosely model the neurons in the brain.

Each artificial neuron receives signals from connected neurons, then processes them and sends a signal to other connected neurons. Picture thousands of these tiny calculators all connected together, each one passing its result to the next layer. 

Layers in a Neural Network

Weights and Biases Explained

Weights and biases are the values a neural network adjusts during training to improve its predictions.

• Weights determine how important each input is. Inputs that are more useful for making a prediction are given higher weights. For example, in spam detection, the word "prize" might have a higher weight than the common word "hello" because it is a stronger indicator of spam.
• Biases help adjust the neuron's decision threshold. They allow a neuron to activate even when the input values are small, making the model more flexible and accurate.

During training, the neural network repeatedly updates its weights and biases based on its errors. Over time, these adjustments help the model make predictions that are closer to the correct answers.

Activation Functions

After a neuron calculates the weighted sum of its inputs and adds the bias, the result is passed through an activation function before moving to the next layer.

An activation function decides whether a neuron should pass its information forward and introduces non-linearity into the network. This allows the model to learn complex patterns and relationships in the data.

Without an activation function, a neural network would behave like a simple linear model, even if it had many hidden layers, making it unable to solve complex problems.