Feature Scaling and Outlier Handling

What Is Feature Scaling?

Feature scaling transforms numerical features to comparable ranges.

Consider a dataset containing:

• Age: 18–70.
• Monthly income: ₹10,000–₹200,000.
• Years of experience: 0–40.

The values use very different numerical ranges. For some algorithms, these differences can affect training or distance calculations.

Feature scaling is particularly useful for methods such as K-nearest neighbors, K-means clustering, support vector machines, and many gradient-based optimization algorithms.

Tree-based algorithms are generally less sensitive to the scale of numerical features.

Standardization

Standardization transforms a numerical feature so that it has a mean of approximately zero and a standard deviation of approximately one, based on the data used to fit the transformation.

The formula is:

[
z = \frac{x-\mu}{\sigma}
]

Where:

• (x) is the original value.
• (\mu) is the feature mean.
• (\sigma) is the feature standard deviation.

Standardization does not guarantee that all transformed values fall between 0 and 1.

In scikit-learn, StandardScaler provides this transformation.

from sklearn.preprocessing import StandardScaler
data = [[20], [30], [40], [50], [60]]
scaler = StandardScaler()
scaled_data = scaler.fit_transform(data)
print(scaled_data)

Min-Max Scaling

Min-max scaling transforms values to a chosen range, commonly 0 to 1.

The formula is:

[
x' = \frac{x-x_{\min}}{x_{\max}-x_{\min}}
]

For example, if the observed minimum is 10 and the maximum is 50, the value 30 becomes:

[
x' = \frac{30-10}{50-10}=0.5
]

In scikit-learn:

from sklearn.preprocessing import MinMaxScaler
data = [[10], [20], [30], [40], [50]]
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(data)
print(scaled_data)

Values outside the range observed during fitting may transform to values below 0 or above 1. Min-max scaling does not automatically protect against extreme outliers.

Understanding Outliers

An outlier is an observation that differs substantially from the general pattern of the data.

For example, a dataset containing most monthly expenses between ₹5,000 and ₹30,000 might include a single value of ₹500,000.

That value could be:

• A data entry error.
• A genuine unusually high expense.
• A special case with a different underlying explanation.

Outliers should be investigated before being removed or modified.

Detecting Outliers with the IQR Method

The interquartile range (IQR) measures the spread of the middle 50% of observations.

[
IQR = Q_3 - Q_1
]

A commonly used rule flags values below:

[
Q_1 - 1.5 \times IQR
]

or above:

[
Q_3 + 1.5 \times IQR
]

as potential outliers.

Python example:

import pandas as pd

values = pd.Series([10, 12, 13, 14, 15, 16, 18, 100])

q1 = values.quantile(0.25)

q3 = values.quantile(0.75)

iqr = q3 - q1

lower_bound = q1 - 1.5 * iqr

upper_bound = q3 + 1.5 * iqr

outliers = values[

(values < lower_bound) | (values > upper_bound)

]

print("Potential outliers:", outliers.tolist())

This rule identifies unusual values for investigation. It does not prove that the values are incorrect.

Avoiding Leakage During Scaling

Like imputation and encoding, scaling should be fitted using the training data only.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

The scaler learns its parameters from X_train and applies them to X_test.

For more complex workflows, a pipeline can manage scaling automatically.