Unsupervised Learning in Machine Learning
Not every dataset contains a target variable. Businesses often collect large amounts of customer, product, and operational information without predefined categories.
Unsupervised learning helps identify patterns within these datasets.
Unsupervised learning is a machine learning approach that identifies structure or relationships in data without using predefined target labels during training.
Common techniques include clustering and dimensionality reduction.
What Is Clustering in Machine Learning?
Clustering groups observations according to a chosen measure of similarity.
For example, an e-commerce company may group customers using:
- Purchase frequency
- Average spending
- Recency of purchases
Customers with similar characteristics may be assigned to the same cluster.
However, clustering algorithms identify mathematical groupings rather than inherently meaningful business segments.
The resulting groups require interpretation and validation.
What Is K-Means Clustering?
K-Means partitions observations into a predefined number of clusters.
The algorithm attempts to minimise the sum of squared distances between observations and their assigned cluster centres.
The objective is:
\[ J=\sum_{i=1}^{n}\lVert x_i-\mu_{c_i}\rVert^2 \]
Here:
- \(x_i\) represents an observation.
- \(\mu_{c_i}\) represents the centre of its assigned cluster.
- \(n\) represents the number of observations.
How Does K-Means Work?
- Select the number of clusters, \(K\).
- Initialise cluster centres.
- Assign observations to their nearest centre.
- Recalculate centres using assigned observations.
- Repeat until the stopping condition is satisfied.
The resulting clusters depend on feature scaling, the selected number of clusters, and initialisation.
K-Means Clustering in Python
import numpy as npfrom sklearn.cluster import KMeansfrom sklearn.preprocessing import StandardScalerX = np.array([ [2, 1500], [3, 2000], [4, 2500], [10, 9000], [12, 11000], [11, 10000]])scaler = StandardScaler()X_scaled = scaler.fit_transform(X)model = KMeans( n_clusters=2, random_state=42, n_init=10)clusters = model.fit_predict(X_scaled)print(clusters)The two features may represent order frequency and total customer spending.
The algorithm assigns each observation to one of two clusters.
Cluster labels are arbitrary identifiers. Label 0 does not inherently mean a low-value or high-value customer.
How Do You Choose the Number of Clusters?
K-Means requires the number of clusters to be specified before fitting.
Two common techniques help evaluate possible values.
Elbow Method
The elbow method compares within-cluster sum of squares across different values of \(K\).
As the number of clusters increases, within-cluster error usually decreases.
A point where improvements begin to diminish may suggest a reasonable cluster count.
However, a clear elbow does not always exist.
Silhouette Score
The silhouette score measures how well an observation fits within its assigned cluster compared with nearby clusters.
For observation \(i\):
\[ s(i)=\frac{b(i)-a(i)}{\max(a(i),b(i))} \]
Here:
- \(a(i)\) is the average distance to observations in the same cluster.
- \(b(i)\) is the smallest average distance to observations in another cluster.
The score ranges from -1 to 1.
Higher values generally indicate better separation under the selected distance metric.
Calculating Silhouette Score
from sklearn.metrics import silhouette_scorescore = silhouette_score( X_scaled, clusters)print(score)Silhouette scores help compare clustering configurations but do not establish that the clusters are meaningful for a business objective.
What Is Hierarchical Clustering?
Hierarchical clustering builds nested groups of observations.
Two common approaches are:
Agglomerative Clustering: Starts with individual observations and repeatedly merges clusters.
Divisive Clustering: Starts with a larger group and recursively divides it.
Hierarchical clustering results can be represented through a dendrogram.
A dendrogram shows how clusters are combined or separated at different similarity levels.
Hierarchical methods can be useful when exploring nested group structure, but they may be computationally expensive for large datasets.
What Is DBSCAN?
DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise.
It identifies clusters using regions of relatively high observation density.
Its main parameters include:
eps: Neighbourhood distance threshold.min_samples: Minimum neighbourhood count used to identify core points.
DBSCAN can identify irregularly shaped clusters and label some observations as noise.
Python Example
from sklearn.cluster import DBSCANmodel = DBSCAN( eps=0.8, min_samples=2)labels = model.fit_predict(X_scaled)print(labels)DBSCAN does not require the number of clusters in advance.
However, its performance depends on the distance metric and density parameters.
It may struggle when clusters have substantially different densities.
Clustering Algorithms Compared
| Algorithm | Main Strength | Limitation |
|---|---|---|
| K-Means | Efficient for compact clusters | Requires predefined K |
| Hierarchical Clustering | Shows nested cluster structure | May be computationally expensive |
| DBSCAN | Identifies arbitrary-shaped clusters and noise | Sensitive to density parameters |
No single clustering algorithm is appropriate for every dataset.
What Is Dimensionality Reduction?
Dimensionality reduction transforms or selects features to represent data using fewer dimensions.
For example, a customer dataset may contain 100 numerical variables.
Some variables may be redundant or strongly related.
Dimensionality reduction can reduce computational requirements and support visualisation.
Two broad approaches are:
Feature Selection: Retains a subset of existing features.
Feature Extraction: Creates new lower-dimensional representations from the original features.
What Is Principal Component Analysis?
Principal Component Analysis (PCA) is a linear dimensionality reduction technique.
It identifies orthogonal directions that capture the greatest variance in centred numerical data.
The first principal component captures the largest possible variance.
Subsequent components capture remaining variance while remaining orthogonal to earlier components.
PCA in Python
import numpy as npfrom sklearn.decomposition import PCAfrom sklearn.preprocessing import StandardScalerX = np.array([ [10, 100, 5], [20, 200, 8], [30, 300, 12], [40, 400, 15], [50, 500, 18]])X_scaled = StandardScaler().fit_transform(X)pca = PCA(n_components=2)X_reduced = pca.fit_transform(X_scaled)print(X_reduced.shape)print(pca.explained_variance_ratio_)The transformation reduces three original features to two principal components.
The explained variance ratio indicates how much of the fitted dataset's variance is represented by each component.
For supervised machine learning, scaling and PCA must be fitted on the training data only.
What Is t-SNE?
t-Distributed Stochastic Neighbor Embedding (t-SNE) is a nonlinear dimensionality reduction technique commonly used for visualising high-dimensional data.
It attempts to preserve certain local neighbourhood relationships in a lower-dimensional representation.
For example, t-SNE may help visualise high-dimensional customer embeddings or image representations.
However, distances between widely separated groups in a t-SNE plot should not automatically be interpreted as accurate original-space distances.
Cluster appearances can also depend on parameter settings.
What Is UMAP?
Uniform Manifold Approximation and Projection (UMAP) is another nonlinear dimensionality reduction method.
It is commonly used to visualise high-dimensional embeddings and explore neighbourhood structure.
UMAP can also serve as a transformation technique, although downstream performance and stability must be evaluated.
Like t-SNE, it does not guarantee that visible groups represent meaningful real-world categories.
PCA vs t-SNE vs UMAP
| Technique | Main Purpose | Key Characteristic |
|---|---|---|
| PCA | Linear dimensionality reduction | Maximises retained variance |
| t-SNE | Nonlinear visualisation | Emphasises local neighbourhood structure |
| UMAP | Nonlinear representation and visualisation | Constructs a graph-based lower-dimensional representation |
PCA is often useful for preprocessing and compression, while t-SNE and UMAP are frequently used for exploratory visualisation.
How Do You Evaluate Unsupervised Learning Models?
Unsupervised models do not always have ground-truth labels for direct evaluation.
Common evaluation methods include:
Silhouette Score: Measures separation and cohesion of clusters.
Davies-Bouldin Index: Evaluates within-cluster similarity relative to separation between clusters.
Calinski-Harabasz Score: Compares between-cluster dispersion with within-cluster dispersion.
Cluster Stability: Examines whether similar groupings appear across resampling or repeated fitting.
Domain Validation: Evaluates whether clusters or representations support meaningful business or scientific interpretation.
An internal metric alone cannot establish that an unsupervised result is useful.
Where Is Unsupervised Learning Used?
| Application | Technique | Objective |
|---|---|---|
| Customer Segmentation | K-Means | Group purchasing patterns |
| Fraud Investigation | Density-based methods | Identify unusual observations |
| Image Analysis | PCA or embeddings | Reduce feature dimensions |
| Document Exploration | Clustering | Group similar content |
| Bioinformatics | Hierarchical clustering | Explore related samples |
| Data Visualisation | t-SNE or UMAP | Examine high-dimensional structure |
Common Unsupervised Learning Mistakes
- Assuming discovered clusters are objectively meaningful categories.
- Using K-Means without appropriate feature scaling.
- Selecting the number of clusters solely from one metric.
- Interpreting t-SNE or UMAP distances as exact original-space distances.
- Assuming dimensionality reduction always improves prediction performance.
- Fitting transformations on test data during supervised evaluation.
Data Science Tutorial for Beginners
Vaishali Ardhana








