NumPy for Data Science
Data science frequently involves mathematical operations on thousands or millions of numerical values. Standard Python lists can store these values, but performing repeated calculations through Python-level loops may become inefficient for large datasets.
NumPy provides specialised data structures and numerical operations designed for efficient array-based computation.
NumPy, short for Numerical Python, is an open-source Python library that supports multidimensional arrays, vectorised operations, broadcasting, linear algebra, and numerical computation.
It forms an important foundation for scientific computing and machine learning libraries.
What Is a NumPy Array?
A NumPy array is an instance of the ndarray class, which stores values in a multidimensional structure.
Unlike general-purpose Python lists, standard NumPy arrays typically contain elements represented using a common data type.
This enables efficient storage and numerical operations.
Consider the following example:
import numpy as npsales = np.array([1200, 1500, 1800, 2200])print(sales)Output:
[1200 1500 1800 2200]The np.array() function converts a compatible Python collection into a NumPy array.
NumPy Arrays vs Python Lists
| Feature | Python List | NumPy Array |
|---|---|---|
| Main Purpose | General-purpose collection | Numerical computation |
| Data Types | Can contain mixed objects | Usually uses a common dtype |
| Element-wise Arithmetic | Requires explicit logic or other tools | Supported directly |
| Multidimensional Data | Nested lists | Native multidimensional arrays |
| Numerical Performance | May require Python-level loops | Optimised array operations |
| Typical Application | General data storage | Mathematical and scientific computing |
NumPy arrays are particularly useful when performing repeated calculations across large numerical datasets.
What Are Array Dimensions and Shapes?
NumPy arrays can contain one or more dimensions.
One-Dimensional Arrays
A one-dimensional array stores values along a single axis.
sales = np.array([1000, 2000, 3000])print(sales.shape)print(sales.ndim)Output:
(3,)
1The shape attribute reports the number of elements along each axis.
The ndim attribute reports the number of dimensions.
Two-Dimensional Arrays
A two-dimensional array represents values arranged in rows and columns.
sales = np.array([ [1000, 1500, 2000], [1200, 1800, 2500]])print(sales.shape)Output:
(2, 3)The array contains two rows and three columns.
A two-dimensional array can represent sales across products and months or numerical features across observations.
How Do NumPy Data Types Work?
NumPy arrays use data types such as int32, int64, float32, and float64.
A data type determines how values are represented and how much memory individual elements occupy.
For example:
values = np.array([10, 20, 30], dtype=np.float64)print(values.dtype)Output:
float64The float64 type generally stores values using 64-bit floating-point representation.
Choosing an appropriate dtype can affect memory usage, numerical precision, and compatibility with other libraries.
What Are Vectorised Operations in NumPy?
Vectorisation allows numerical operations to be applied directly to arrays without explicitly writing a Python loop for every element.
Consider a retailer increasing product prices by 5%.
import numpy as npprices = np.array([1000, 1500, 2000, 2500])updated_prices = prices * 1.05print(updated_prices)Output:
[1050. 1575. 2100. 2625.]NumPy applies multiplication element-wise across the array.
This approach is generally more concise and can be substantially faster than equivalent Python-level loops for large numerical arrays.
Element-Wise Arithmetic
NumPy supports common arithmetic operations.
a = np.array([10, 20, 30])b = np.array([2, 4, 5])print(a + b)print(a - b)print(a * b)print(a / b)Output:
[12 24 35]
[ 8 16 25]
[ 20 80 150]
[5. 5. 6.]These operations process corresponding elements of compatible arrays.
What Are Important NumPy Functions?
NumPy provides built-in functions for common numerical calculations.
| Function | Purpose |
|---|---|
np.array() |
Creates an array |
np.mean() |
Calculates the arithmetic mean |
np.median() |
Calculates the median |
np.std() |
Calculates standard deviation |
np.var() |
Calculates variance |
np.sum() |
Calculates the total |
np.min() |
Returns the minimum value |
np.max() |
Returns the maximum value |
np.reshape() |
Changes array dimensions |
np.dot() |
Computes dot products and related contractions |
Consider a dataset containing daily revenue.
revenue = np.array([1200, 1500, 2000, 2500, 3000])print("Mean:", np.mean(revenue))print("Median:", np.median(revenue))print("Total:", np.sum(revenue))Output:
Mean: 2040.0
Median: 2000.0
Total: 10200NumPy calculates these values directly from the array.
By default, np.std() and np.var() use a population-style denominator of \(N\). A sample calculation using \(N-1\) can be requested with ddof=1.
What Is Broadcasting in NumPy?
Broadcasting allows NumPy to perform arithmetic on arrays with different but compatible shapes.
Consider a two-dimensional sales array.
sales = np.array([ [1000, 1500, 2000], [1200, 1800, 2500]])bonus = np.array([100, 200, 300])updated_sales = sales + bonusprint(updated_sales)Output:
[[1100 1700 2300]
[1300 2000 2800]]The one-dimensional array is broadcast across the rows of the two-dimensional array.
Broadcasting follows shape compatibility rules. Two dimensions are compatible when they are equal or when one of them is 1.
This allows concise mathematical operations without manually duplicating data.
What Is Array Reshaping?
Reshaping changes the dimensions of an array while preserving its total number of elements.
values = np.array([10, 20, 30, 40, 50, 60])matrix = values.reshape(2, 3)print(matrix)Output:
[[10 20 30]
[40 50 60]]The original array contains six elements.
The reshaped array contains two rows and three columns.
Reshaping is important when preparing numerical inputs for machine learning models, which frequently expect a feature matrix with a defined number of observations and features.
How Does NumPy Support Linear Algebra?
Machine learning algorithms frequently use vectors and matrices to represent numerical information.
A dataset containing 100 observations and five input features can be represented as a matrix with shape (100, 5).
Matrix multiplication is used to combine input features with model parameters.
Consider:
X = np.array([ [1, 2], [3, 4]])weights = np.array([0.5, 1.5])predictions = X @ weightsprint(predictions)Output:
[3.5 7.5]The @ operator performs matrix multiplication.
Mathematically, a linear model can be expressed as:
\[ \hat{y}=X\beta \]
Here, \(X\) represents the feature matrix and \(\beta\) represents the coefficient vector.
NumPy supports the numerical operations required to evaluate such expressions.
How Is NumPy Used in Machine Learning?
NumPy supports several fundamental machine learning operations.
Feature Representation: Numerical features are organised into matrices and vectors.
Statistical Calculations: Means, variances, and standard deviations help describe datasets.
Numerical Transformations: Features can be normalised or transformed using array operations.
Linear Algebra: Matrix multiplication and decomposition support regression and other algorithms.
Optimisation: Numerical operations support gradient calculations and parameter updates.
For example, a model's prediction error can be measured using Mean Squared Error.
\[ MSE=\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \]
NumPy implementation:
actual = np.array([100, 150, 200])predicted = np.array([110, 140, 190])mse = np.mean((actual - predicted) ** 2)print(mse)Output:
100.0This calculation illustrates how NumPy performs element-wise subtraction, squaring, and aggregation.
Common Mistakes Beginners Should Avoid
- Confusing element-wise multiplication with matrix multiplication:
*performs element-wise multiplication, while@performs matrix multiplication. - Using incompatible array shapes: Broadcasting requires compatible dimensions.
- Ignoring dtype behaviour: Integer or floating-point representation can affect numerical results.
- Reshaping with an incorrect element count: The new shape must contain the same total number of elements.
- Assuming arrays always copy data: Some indexing and reshaping operations return views that share underlying memory.
Data Science Tutorial for Beginners
Vaishali Ardhana








