Machine Learning Fundamentals
Traditional programming relies on explicitly defined instructions. Machine learning uses historical data to learn relationships that can be applied to new observations.
For example, an online retailer can train a model using previous customer transactions to estimate which customers are likely to stop purchasing.
Machine learning is a branch of artificial intelligence that develops mathematical models capable of learning patterns from data to make predictions or support decisions.
Machine learning models use numerical features, algorithms, and optimisation methods to identify relationships relevant to a specified task.
How Does Machine Learning Work?
A typical machine learning process involves five stages:
- Data Preparation: Collect, clean, and transform relevant records.
- Feature Selection: Identify input variables containing useful information.
- Model Training: Fit an algorithm using historical observations.
- Model Evaluation: Measure performance using unseen data.
- Prediction: Apply the trained model to new observations.
A supervised model can be represented mathematically as:
\[ \hat{y}=f(X;\theta) \]
Here:
- \(X\) represents input features.
- \(\theta\) represents learned model parameters.
- \(f\) represents the prediction function.
- \(\hat{y}\) represents the predicted output.
For example, a delivery time prediction model may use distance, traffic conditions, and scheduled stops as input features.
The model learns a relationship between these variables and historical delivery durations.
What Are the Main Types of Machine Learning?
Machine learning is commonly classified into supervised, unsupervised, and reinforcement learning. Self-supervised learning is also important in modern representation learning.
| Type | Training Information | Objective | Example |
|---|---|---|---|
| Supervised Learning | Labelled observations | Predict target values | Fraud detection |
| Unsupervised Learning | Unlabelled observations | Identify patterns or structure | Customer segmentation |
| Reinforcement Learning | Rewards from interactions | Learn actions that maximise expected reward | Sequential decision-making |
| Self-Supervised Learning | Targets derived from data | Learn useful representations | Language model pretraining |
Supervised Learning
Supervised learning uses datasets containing input features and known target values.
For example:
| Age | Monthly Spending (₹) | Previous Orders | Churn |
|---|---|---|---|
| 25 | 5000 | 8 | 0 |
| 30 | 2000 | 2 | 1 |
| 35 | 7500 | 12 | 0 |
| 28 | 1500 | 1 | 1 |
The model learns relationships between customer attributes and churn outcomes.
Supervised learning includes two major problem types.
Regression: Predicts continuous numerical values.
Classification: Predicts categories or class probabilities.
Unsupervised Learning
Unsupervised learning identifies structure without predefined target labels.
For example, a retailer may group customers according to spending and purchase frequency.
Algorithms such as K-Means and DBSCAN can identify clusters of observations with similar characteristics.
Reinforcement Learning
Reinforcement learning trains an agent to select actions through interactions with an environment.
The agent receives rewards or penalties associated with its actions.
The objective is to learn a policy that maximises expected cumulative reward.
Applications include robotics, game-playing systems, and certain sequential optimisation problems.
What Is the Difference Between Features and Labels?
A feature is an input variable used to generate a prediction.
A label or target is the outcome a supervised learning model attempts to predict.
Consider a house price dataset.
| Area (sq. ft.) | Bedrooms | Property Age | Price (₹) |
|---|---|---|---|
| 1000 | 2 | 5 | 4500000 |
| 1500 | 3 | 8 | 6500000 |
| 2000 | 4 | 3 | 9000000 |
Area, bedrooms, and property age are features.
Price is the target variable.
The feature matrix is represented by:
\[ X\in\mathbb{R}^{n\times p} \]
Here, \(n\) is the number of observations and \(p\) is the number of features.
The target vector is:
\[ y\in\mathbb{R}^{n} \]
What Are Training, Validation, and Test Datasets?
A machine learning dataset is commonly divided into separate subsets.
| Dataset | Purpose |
|---|---|
| Training Set | Fits model parameters |
| Validation Set | Selects models and tunes hyperparameters |
| Test Set | Evaluates final performance on held-out data |
A common illustrative split is 70% training, 15% validation, and 15% testing.
However, the appropriate division depends on dataset size, model requirements, and observation structure.
Train-Test Split in Python
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42)The code reserves 20% of observations for testing.
Random splitting is appropriate only when it matches the intended prediction scenario.
Time-series forecasting usually requires chronological splitting, while datasets containing repeated records for individuals may require group-aware splitting.
What Are Model Parameters and Hyperparameters?
Parameters are values learned during model training.
For example, linear regression learns coefficients representing relationships between features and predictions.
Hyperparameters are settings selected outside the main parameter-fitting process.
| Parameters | Hyperparameters |
|---|---|
| Learned from training data | Configured or tuned |
| Regression coefficients | Learning rate |
| Neural network weights | Tree maximum depth |
| Model intercepts | Number of neighbours |
Hyperparameters are commonly selected through validation or cross-validation.
The test set should remain separate from the tuning process.
What Are Overfitting and Underfitting?
Overfitting
Overfitting occurs when a model learns dataset-specific patterns or noise that do not generalise well.
A highly complex model may achieve excellent training performance but poor test performance.
Underfitting
Underfitting occurs when a model is too limited to capture important patterns.
It may perform poorly on both training and test data.
| Condition | Training Performance | Test Performance |
|---|---|---|
| Underfitting | Poor | Poor |
| Reasonable Fit | Good | Similar and good |
| Overfitting | Very good | Substantially worse |
Regularisation, suitable model complexity, and reliable validation can help manage these problems.
What Is the Bias-Variance Trade-Off?
Bias describes systematic error caused by restrictive modelling assumptions.
Variance describes sensitivity to changes in the training data.
A simple model may have high bias, while an overly flexible model may have high variance.
The goal is to select model complexity that achieves reliable generalisation rather than minimising training error alone.
How Do You Train a Basic Machine Learning Model?
Scikit-learn provides a consistent interface for model development.
import numpy as npfrom sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LinearRegressionX = np.array([ [1000], [1500], [2000], [2500], [3000], [3500]])y = np.array([ 3000000, 4500000, 6000000, 7500000, 9000000, 10500000])X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.33, random_state=42)model = LinearRegression()model.fit(X_train, y_train)prediction = model.predict([[2200]])print(prediction)Output:
[6600000.]The model estimates a property price of ₹66 lakh for an area of 2,200 square feet.
The example uses an intentionally simple relationship. Real property prices depend on several additional factors.
Common Machine Learning Mistakes
- Training and evaluating a model on the same observations.
- Applying preprocessing before splitting data.
- Using future information as an input feature.
- Assuming complex algorithms always perform better.
- Evaluating models using metrics unrelated to the actual objective.
Data Science Tutorial for Beginners
Vaishali Ardhana








