Menu

Linear Algebra for Data Science

Machine learning algorithms frequently process datasets containing hundreds or thousands of observations, each described by multiple numerical features.

Linear algebra provides the mathematical framework for representing this information as vectors and matrices.

Linear algebra is the branch of mathematics concerned with vectors, matrices, linear transformations, and systems of linear equations. In data science, these concepts support numerical computation, regression, dimensionality reduction, recommendation systems, and neural network operations.

For example, a customer dataset containing age, income, and purchase frequency can be represented as a matrix. Machine learning models process the matrix using mathematical operations rather than calculating every observation independently.

What Is a Vector in Linear Algebra?

A vector is an ordered collection of numerical values.

Vectors are commonly represented as column matrices.

For example:

\[ v= \begin{bmatrix} 10\\ 20\\ 30 \end{bmatrix} \]

This is a three-dimensional vector containing three components.

In data science, vectors can represent individual observations, model parameters, or numerical features.

Consider a customer represented by three attributes:

  • Age: 25
  • Monthly spending: ₹5,000
  • Previous orders: 8

The feature vector can be written as:

\[ x= \begin{bmatrix} 25\\ 5000\\ 8 \end{bmatrix} \]

The values together describe one customer.

Creating Vectors Using NumPy

import numpy as npcustomer = np.array([25, 5000, 8])print(customer)print(customer.shape)

Output:

[  25 5000    8]
(3,)

NumPy represents this example as a one-dimensional array with three elements.

What Are the Main Vector Operations?

Vectors support mathematical operations such as addition, scalar multiplication, and dot products.

Vector Addition

Two vectors of equal dimensions can be added element-wise.

\[ a= \begin{bmatrix} 2\\ 4 \end{bmatrix}, \quad b= \begin{bmatrix} 3\\ 5 \end{bmatrix} \]

Then:

\[ a+b= \begin{bmatrix} 5\\ 9 \end{bmatrix} \]

Scalar Multiplication

Scalar multiplication multiplies every component of a vector by a constant.

\[ 3 \begin{bmatrix} 2\\ 4 \end{bmatrix} = \begin{bmatrix} 6\\ 12 \end{bmatrix} \]

This operation is useful for adjusting numerical features or applying model coefficients.

Dot Product

The dot product combines two equal-length vectors into a scalar.

For vectors \(a\) and \(b\):

\[ a\cdot b=\sum_{i=1}^{n}a_ib_i \]

Consider:

\[ a=[2,3], \quad b=[4,5] \]

\[ a\cdot b=(2\times4)+(3\times5) \]

\[ a\cdot b=23 \]

Python implementation:

import numpy as npa = np.array([2, 3])b = np.array([4, 5])result = np.dot(a, b)print(result)

Output:

23

Dot products are used in linear regression, similarity calculations, and neural network computations.

What Is a Matrix in Linear Algebra?

A matrix is a rectangular arrangement of values organised into rows and columns.

For example:

\[ A= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix} \]

This matrix has two rows and three columns.

Its dimensions are written as:

\[ A\in\mathbb{R}^{2\times3} \]

In data science, a dataset containing \(n\) observations and \(p\) features is often represented as:

\[ X\in\mathbb{R}^{n\times p} \]

For example, a dataset with 1,000 customers and five numerical features has dimensions:

\[ X\in\mathbb{R}^{1000\times5} \]

Each row represents an observation, while each column represents a feature.

Creating a Matrix Using NumPy

import numpy as npdata = np.array([    [25, 5000, 8],    [30, 6500, 12],    [22, 3000, 4]])print(data.shape)

Output:

(3, 3)

The array contains three customer records and three features.

What Are the Main Matrix Operations?

Matrix Addition

Matrix addition combines corresponding elements of matrices with identical dimensions.

\[ A= \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix} \]

\[ B= \begin{bmatrix} 5&6\\ 7&8 \end{bmatrix} \]

Then:

\[ A+B= \begin{bmatrix} 6&8\\ 10&12 \end{bmatrix} \]

Matrix addition is commonly used in numerical transformations and intermediate machine learning calculations.

Matrix Multiplication

Matrix multiplication combines rows of the first matrix with columns of the second.

For matrices:

\[ A\in\mathbb{R}^{m\times n} \]

\[ B\in\mathbb{R}^{n\times p} \]

Their product has dimensions:

\[ AB\in\mathbb{R}^{m\times p} \]

The inner dimensions must match.

Consider:

\[ A= \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix} \]

\[ B= \begin{bmatrix} 5&6\\ 7&8 \end{bmatrix} \]

Their product is:

\[ AB= \begin{bmatrix} 19&22\\ 43&50 \end{bmatrix} \]

Python implementation:

import numpy as npA = np.array([    [1, 2],    [3, 4]])B = np.array([    [5, 6],    [7, 8]])result = A @ Bprint(result)

Output:

[[19 22]
 [43 50]]

The @ operator performs matrix multiplication.

Matrix Transposition

Transposition exchanges the rows and columns of a matrix.

For example:

\[ A= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix} \]

Its transpose is:

\[ A^T= \begin{bmatrix} 1&4\\ 2&5\\ 3&6 \end{bmatrix} \]

Python example:

A = np.array([    [1, 2, 3],    [4, 5, 6]])print(A.T)

Output:

[[1 4]
 [2 5]
 [3 6]]

Transposition is frequently used in covariance calculations, regression, and matrix transformations.

What Is the Identity Matrix?

An identity matrix is a square matrix containing ones along its main diagonal and zeros elsewhere.

For example:

\[ I= \begin{bmatrix} 1&0\\ 0&1 \end{bmatrix} \]

Multiplying a compatible matrix by an identity matrix leaves it unchanged.

\[ AI=A \]

Identity matrices are important in linear algebra operations, regularisation, and solving systems of equations.

What Is Matrix Inversion?

An inverse matrix reverses the effect of multiplication by an invertible square matrix.

For a matrix \(A\):

\[ AA^{-1}=I \]

Not every matrix has an inverse.

A square matrix must be nonsingular to have an inverse.

Matrix inversion appears in theoretical formulations of regression and other mathematical problems.

However, numerical applications generally prefer solving linear systems directly rather than explicitly calculating matrix inverses.

How Is Linear Algebra Used in Linear Regression?

Multiple linear regression predicts a numerical outcome using several input features.

The matrix representation is:

\[ \hat{y}=X\beta+b \]

Here:

  • \(X\) represents the feature matrix.
  • \(\beta\) represents model coefficients.
  • \(b\) represents the intercept.
  • \(\hat{y}\) represents predictions.

Consider a model that predicts spending using age and previous orders.

Each customer's features form a row in \(X\).

The model multiplies these features by the learned coefficient vector to generate predictions.

NumPy implementation:

import numpy as npX = np.array([    [25, 8],    [30, 12],    [22, 4]])weights = np.array([20, 150])intercept = 500predictions = X @ weights + interceptprint(predictions)

Output:

[2200 2900 1540]

These are predictions from illustrative coefficients, not values learned by fitting a model.

The calculation demonstrates how matrix multiplication allows multiple predictions to be generated together.

How Is Linear Algebra Used in Neural Networks?

Neural networks process numerical information through layers of mathematical transformations.

A basic neural network layer can be expressed as:

\[ Z=XW+b \]

Here:

  • \(X\) represents input features.
  • \(W\) represents the weight matrix.
  • \(b\) represents the bias vector.
  • \(Z\) represents the linear transformation output.

An activation function is then applied:

\[ A=f(Z) \]

The activation function introduces nonlinear behaviour into the network.

Without nonlinear activation functions, a sequence of linear transformations would remain equivalent to a single linear transformation.

Linear algebra therefore supports the numerical operations performed during neural network inference and training.

Where Else Is Linear Algebra Used in Data Science?

Application Role of Linear Algebra
Linear Regression Combines features and coefficients
Neural Networks Processes weighted inputs across layers
Recommendation Systems Represents users and products as vectors
Principal Component Analysis Identifies directions of variation
Computer Vision Represents image information using arrays and transformations
Natural Language Processing Represents text using numerical embeddings

Common Mistakes in Linear Algebra

  1. Confusing element-wise multiplication with matrix multiplication.
  2. Ignoring dimension compatibility during matrix operations.
  3. Assuming every square matrix has an inverse.
  4. Confusing rows representing observations with columns representing features.
  5. Applying matrix operations without checking whether their mathematical assumptions are satisfied.

Data Science Tutorial for Beginners

VA

Vaishali Ardhana



Learn via Video Course

undefined Logo

0 hrs



Get in Touch For Details