Scaling and Normalizing Numerical Features

Numerical features can have very different scales. For example, annual income might range from thousands to millions, while customer ratings range from 1 to 5.

Some machine learning algorithms are sensitive to these differences. Scaling transforms numerical values into a more suitable range or distribution.

Standardization

Standardization transforms a feature by subtracting its mean and dividing by its standard deviation.

[
z = \frac{x-\mu}{\sigma}
]

Here, (x) is the original value, (\mu) is the mean, and (\sigma) is the standard deviation.

import pandas as pd

from sklearn.preprocessing import StandardScaler

df = pd.DataFrame({

"Income": [30000, 50000, 70000, 90000],

"Age": [22, 30, 40, 50]

})

scaler = StandardScaler()

scaled_values = scaler.fit_transform(df)

scaled_df = pd.DataFrame(

scaled_values,

columns=df.columns

)

print(scaled_df)

After standardization, each feature has approximately zero mean and unit variance on the data used to fit the scaler, provided the feature has nonzero variance.

Standardization is commonly useful for algorithms such as logistic regression, support vector machines, and distance-based methods.

Min-Max Scaling

Min-max scaling transforms values into a specified range, usually between 0 and 1.

[
x' = \frac{x-x_{\min}}{x_{\max}-x_{\min}}
]

import pandas as pd

from sklearn.preprocessing import MinMaxScaler

df = pd.DataFrame({

"Age": [20, 30, 40, 50, 60]

})

scaler = MinMaxScaler()

df["Scaled_Age"] = scaler.fit_transform(df[["Age"]])

print(df)

Min-max scaling preserves the relative ordering of values, but extreme observations can strongly affect the resulting range.

Avoiding Data Leakage During Scaling

A common mistake is fitting a scaler on the complete dataset before separating training and testing data. This allows information from the test set to influence preprocessing.

Instead, fit the scaler using only the training data and apply the same transformation to the test data.

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

import pandas as pd

X = pd.DataFrame({

"Income": [30000, 45000, 50000, 65000, 80000, 95000],

"Age": [22, 28, 30, 35, 42, 50]

})

y = [0, 0, 0, 1, 1, 1]

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.33, random_state=42

)

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

X_test_scaled = scaler.transform(X_test)

The scaler learns its parameters from X_train only. Using transform() on X_test applies those learned parameters without fitting the scaler again.