Feature Selection and Handling Imbalanced Data

A dataset may contain many features, but not all of them contribute useful information. Some features may be redundant, irrelevant, or strongly related to the target in ways that will not generalize to new data.

Feature selection helps identify a smaller, more useful set of input variables.

Removing Irrelevant Columns

Consider a customer dataset with an identifier, age, annual spending, and purchase count.

import pandas as pd

df = pd.DataFrame({

"Customer_ID": [101, 102, 103, 104],

"Age": [25, 35, 45, 30],

"Annual_Spending": [15000, 30000, 45000, 22000],

"Purchases": [5, 10, 15, 7]

})

X = df[["Age", "Annual_Spending", "Purchases"]]

print(X)

A unique identifier is often excluded because it does not necessarily represent a meaningful predictive pattern. However, whether a column should be removed depends on its purpose and the data-generating process.

Examining Correlated Features

Highly correlated features may contain overlapping information. Correlation analysis can help identify such relationships.

import pandas as pd

df = pd.DataFrame({

"Study_Hours": [1, 2, 3, 4, 5],

"Practice_Hours": [2, 4, 6, 8, 10],

"Exam_Score": [40, 50, 60, 75, 85]

})

print(df.corr(numeric_only=True))

Strong correlation between two features does not automatically mean that one should be removed. Consider domain knowledge, model requirements, and validation results before selecting features.

Understanding Imbalanced Classification Data

In classification problems, one class may appear much more frequently than another. For example, a fraud dataset might contain many legitimate transactions and very few fraudulent ones.

import pandas as pd

df = pd.DataFrame({

"Transaction": range(1, 11),

"Fraud": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]

})

print(df["Fraud"].value_counts())

print(df["Fraud"].value_counts(normalize=True) * 100)

This shows the count and percentage of each class. An imbalanced dataset can make accuracy misleading because a model may predict the majority class most of the time while failing to detect important minority cases.

Possible approaches include class weighting, oversampling, and undersampling. These techniques should be applied carefully and only to training data, with performance evaluated using appropriate measures such as precision, recall, F1-score, and precision-recall curves.