Menu

Unsupervised Learning in Machine Learning

Not every dataset contains a target variable. Businesses often collect large amounts of customer, product, and operational information without predefined categories.

Unsupervised learning helps identify patterns within these datasets.

Unsupervised learning is a machine learning approach that identifies structure or relationships in data without using predefined target labels during training.

Common techniques include clustering and dimensionality reduction.

What Is Clustering in Machine Learning?

Clustering groups observations according to a chosen measure of similarity.

For example, an e-commerce company may group customers using:

  • Purchase frequency
  • Average spending
  • Recency of purchases

Customers with similar characteristics may be assigned to the same cluster.

However, clustering algorithms identify mathematical groupings rather than inherently meaningful business segments.

The resulting groups require interpretation and validation.

What Is K-Means Clustering?

K-Means partitions observations into a predefined number of clusters.

The algorithm attempts to minimise the sum of squared distances between observations and their assigned cluster centres.

The objective is:

\[ J=\sum_{i=1}^{n}\lVert x_i-\mu_{c_i}\rVert^2 \]

Here:

  • \(x_i\) represents an observation.
  • \(\mu_{c_i}\) represents the centre of its assigned cluster.
  • \(n\) represents the number of observations.

How Does K-Means Work?

  1. Select the number of clusters, \(K\).
  2. Initialise cluster centres.
  3. Assign observations to their nearest centre.
  4. Recalculate centres using assigned observations.
  5. Repeat until the stopping condition is satisfied.

The resulting clusters depend on feature scaling, the selected number of clusters, and initialisation.

K-Means Clustering in Python

import numpy as npfrom sklearn.cluster import KMeansfrom sklearn.preprocessing import StandardScalerX = np.array([    [2, 1500],    [3, 2000],    [4, 2500],    [10, 9000],    [12, 11000],    [11, 10000]])scaler = StandardScaler()X_scaled = scaler.fit_transform(X)model = KMeans(    n_clusters=2,    random_state=42,    n_init=10)clusters = model.fit_predict(X_scaled)print(clusters)

The two features may represent order frequency and total customer spending.

The algorithm assigns each observation to one of two clusters.

Cluster labels are arbitrary identifiers. Label 0 does not inherently mean a low-value or high-value customer.

How Do You Choose the Number of Clusters?

K-Means requires the number of clusters to be specified before fitting.

Two common techniques help evaluate possible values.

Elbow Method

The elbow method compares within-cluster sum of squares across different values of \(K\).

As the number of clusters increases, within-cluster error usually decreases.

A point where improvements begin to diminish may suggest a reasonable cluster count.

However, a clear elbow does not always exist.

Silhouette Score

The silhouette score measures how well an observation fits within its assigned cluster compared with nearby clusters.

For observation \(i\):

\[ s(i)=\frac{b(i)-a(i)}{\max(a(i),b(i))} \]

Here:

  • \(a(i)\) is the average distance to observations in the same cluster.
  • \(b(i)\) is the smallest average distance to observations in another cluster.

The score ranges from -1 to 1.

Higher values generally indicate better separation under the selected distance metric.

Calculating Silhouette Score

from sklearn.metrics import silhouette_scorescore = silhouette_score(    X_scaled,    clusters)print(score)

Silhouette scores help compare clustering configurations but do not establish that the clusters are meaningful for a business objective.

What Is Hierarchical Clustering?

Hierarchical clustering builds nested groups of observations.

Two common approaches are:

Agglomerative Clustering: Starts with individual observations and repeatedly merges clusters.

Divisive Clustering: Starts with a larger group and recursively divides it.

Hierarchical clustering results can be represented through a dendrogram.

A dendrogram shows how clusters are combined or separated at different similarity levels.

Hierarchical methods can be useful when exploring nested group structure, but they may be computationally expensive for large datasets.

What Is DBSCAN?

DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise.

It identifies clusters using regions of relatively high observation density.

Its main parameters include:

  • eps: Neighbourhood distance threshold.
  • min_samples: Minimum neighbourhood count used to identify core points.

DBSCAN can identify irregularly shaped clusters and label some observations as noise.

Python Example

from sklearn.cluster import DBSCANmodel = DBSCAN(    eps=0.8,    min_samples=2)labels = model.fit_predict(X_scaled)print(labels)

DBSCAN does not require the number of clusters in advance.

However, its performance depends on the distance metric and density parameters.

It may struggle when clusters have substantially different densities.

Clustering Algorithms Compared

Algorithm Main Strength Limitation
K-Means Efficient for compact clusters Requires predefined K
Hierarchical Clustering Shows nested cluster structure May be computationally expensive
DBSCAN Identifies arbitrary-shaped clusters and noise Sensitive to density parameters

No single clustering algorithm is appropriate for every dataset.

What Is Dimensionality Reduction?

Dimensionality reduction transforms or selects features to represent data using fewer dimensions.

For example, a customer dataset may contain 100 numerical variables.

Some variables may be redundant or strongly related.

Dimensionality reduction can reduce computational requirements and support visualisation.

Two broad approaches are:

Feature Selection: Retains a subset of existing features.

Feature Extraction: Creates new lower-dimensional representations from the original features.

What Is Principal Component Analysis?

Principal Component Analysis (PCA) is a linear dimensionality reduction technique.

It identifies orthogonal directions that capture the greatest variance in centred numerical data.

The first principal component captures the largest possible variance.

Subsequent components capture remaining variance while remaining orthogonal to earlier components.

PCA in Python

import numpy as npfrom sklearn.decomposition import PCAfrom sklearn.preprocessing import StandardScalerX = np.array([    [10, 100, 5],    [20, 200, 8],    [30, 300, 12],    [40, 400, 15],    [50, 500, 18]])X_scaled = StandardScaler().fit_transform(X)pca = PCA(n_components=2)X_reduced = pca.fit_transform(X_scaled)print(X_reduced.shape)print(pca.explained_variance_ratio_)

The transformation reduces three original features to two principal components.

The explained variance ratio indicates how much of the fitted dataset's variance is represented by each component.

For supervised machine learning, scaling and PCA must be fitted on the training data only.

What Is t-SNE?

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a nonlinear dimensionality reduction technique commonly used for visualising high-dimensional data.

It attempts to preserve certain local neighbourhood relationships in a lower-dimensional representation.

For example, t-SNE may help visualise high-dimensional customer embeddings or image representations.

However, distances between widely separated groups in a t-SNE plot should not automatically be interpreted as accurate original-space distances.

Cluster appearances can also depend on parameter settings.

What Is UMAP?

Uniform Manifold Approximation and Projection (UMAP) is another nonlinear dimensionality reduction method.

It is commonly used to visualise high-dimensional embeddings and explore neighbourhood structure.

UMAP can also serve as a transformation technique, although downstream performance and stability must be evaluated.

Like t-SNE, it does not guarantee that visible groups represent meaningful real-world categories.

PCA vs t-SNE vs UMAP

Technique Main Purpose Key Characteristic
PCA Linear dimensionality reduction Maximises retained variance
t-SNE Nonlinear visualisation Emphasises local neighbourhood structure
UMAP Nonlinear representation and visualisation Constructs a graph-based lower-dimensional representation

PCA is often useful for preprocessing and compression, while t-SNE and UMAP are frequently used for exploratory visualisation.

How Do You Evaluate Unsupervised Learning Models?

Unsupervised models do not always have ground-truth labels for direct evaluation.

Common evaluation methods include:

Silhouette Score: Measures separation and cohesion of clusters.

Davies-Bouldin Index: Evaluates within-cluster similarity relative to separation between clusters.

Calinski-Harabasz Score: Compares between-cluster dispersion with within-cluster dispersion.

Cluster Stability: Examines whether similar groupings appear across resampling or repeated fitting.

Domain Validation: Evaluates whether clusters or representations support meaningful business or scientific interpretation.

An internal metric alone cannot establish that an unsupervised result is useful.

Where Is Unsupervised Learning Used?

Application Technique Objective
Customer Segmentation K-Means Group purchasing patterns
Fraud Investigation Density-based methods Identify unusual observations
Image Analysis PCA or embeddings Reduce feature dimensions
Document Exploration Clustering Group similar content
Bioinformatics Hierarchical clustering Explore related samples
Data Visualisation t-SNE or UMAP Examine high-dimensional structure

Common Unsupervised Learning Mistakes

  1. Assuming discovered clusters are objectively meaningful categories.
  2. Using K-Means without appropriate feature scaling.
  3. Selecting the number of clusters solely from one metric.
  4. Interpreting t-SNE or UMAP distances as exact original-space distances.
  5. Assuming dimensionality reduction always improves prediction performance.
  6. Fitting transformations on test data during supervised evaluation.

Data Science Tutorial for Beginners

VA

Vaishali Ardhana



Learn via Video Course

undefined Logo

0 hrs



Get in Touch For Details