Normalization vs Standardization

Normalization and standardization are both feature-scaling techniques, but they work differently and are used in different situations.

Normalization

Normalization rescales data into a fixed range, usually between 0 and 1. It is most useful when your data does not follow a normal distribution, or when you know the exact minimum and maximum values in advance.

Standardization

Standardization rescales data to have a mean of 0 and a standard deviation of 1. It works best when your data roughly follows a normal distribution and is less affected by outliers than normalization.

Key differences

• Normalization compresses data into a fixed range, while standardization centres it around zero.
• Normalization is sensitive to outliers because extreme values affect the minimum and maximum. Standardization handles outliers better.
• Normalization is commonly used in algorithms like KNN and neural networks. Standardization is commonly used in algorithms like logistic regression and SVM.

Choosing between the two depends on your dataset's distribution and the algorithm you plan to use, since picking the wrong method can affect model accuracy.