Regularization Techniques Explained

Regularization is a set of techniques used to prevent a model from overfitting, by discouraging it from relying too heavily on any single feature.

It works by adding a penalty term to the model's loss function, which increases whenever the model assigns very large weights to specific inputs.

a. L1 Regularization, also known as Lasso, can shrink some feature weights all the way to zero, effectively removing those features from the model entirely.

b. L2 Regularization, also known as Ridge, shrinks feature weights closer to zero without eliminating them completely, keeping every feature in play but reducing extreme influence.