Data Cleaning and Handling Missing Values

After understanding the dataset structure, the next step is to clean the data. In real-world student datasets, some columns may contain missing values, which can affect analysis and model performance if not handled properly.

Checking For Missing Values

First, we check whether any columns contain missing values.

df.isnull().sum()

This command displays the total number of missing values in each column, helping us identify which features require cleaning.

Filling Missing Values In Categorical Columns

In this project, missing values are found in specific categorical columns. Instead of removing rows, we fill the missing entries using the most frequent value (mode) of each column. This approach maintains dataset size while preserving meaningful information.

# Handle missing values by filling with the most frequent value for categorical columns

categorical_cols_with_na = ['Teacher_Quality', 'Parental_Education_Level', 'Distance_from_Home']

for col in categorical_cols_with_na:

most_frequent = df[col].mode()[0]

df[col].fillna(most_frequent, inplace=True)

# Verify that missing values are handled

df[categorical_cols_with_na].isnull().sum()

By replacing missing values with the most frequent category, we ensure consistency in the dataset without introducing unrealistic data.

After this step, the dataset becomes clean and ready for exploratory data analysis and visualization in the next lesson.