Handling Missing Values and Random Data

Real-world datasets can contain missing or invalid values. NumPy provides special values and functions that help identify and handle some of these situations.

Understanding NaN

NaN stands for “Not a Number” and is commonly used to represent missing or undefined numerical values.

import numpy as np

measurements = np.array([12.5, 15.0, np.nan, 18.5, 20.0])

print(np.isnan(measurements))

A normal mean calculation involving NaN generally returns NaN.

print(np.mean(measurements))

print(np.nanmean(measurements))

np.nanmean() calculates the mean while ignoring NaN values. However, this should only be done when ignoring missing observations is appropriate for the analysis. If all values are missing, the function cannot produce a meaningful mean.

Replacing Missing Values

You can create a Boolean mask and replace missing values with a chosen value.

measurements = np.array([12.5, 15.0, np.nan, 18.5, 20.0])

cleaned = np.where(

np.isnan(measurements),

0,

measurements

)

print(cleaned)

Here, missing values are replaced with zero. In real projects, this choice depends on the meaning of the data; zero may represent a real measurement rather than missing information. Other strategies include using the mean, median, or a domain-specific replacement.

Generating Random Numbers

NumPy's random number generator is useful for simulations, sampling, and creating synthetic datasets.

rng = np.random.default_rng(42)
sales = rng.integers(1000, 5000, size=5)
ratings = rng.uniform(3.0, 5.0, size=5)
print(sales)
print(ratings)

The integer range includes the lower bound but excludes the upper bound. The uniform() function generates floating-point values within the specified range. Using a fixed seed, such as 42, makes the generated sequence reproducible.