Machine learning models tend to overfit the training data and cannot generalise well to unseen data. This is known as overfitting. To solve this, machine learning engineers use regularisation techniques that control model complexity and improve generalisation.<\/p>\n\n\n\n
Lasso Regression is among the most common regularisation techniques in machine learning. This reduces overfitting and also does automatic feature selection, making models simpler and more efficient.<\/p>\n\n\n\n
In this article, you will learn what Lasso Regression is, how L1 regularisation works, why feature selection matters, how the lambda parameter affects the model performance and how to implement Lasso Regression using Scikit Learn.<\/p>\n\n\n\n
\n Lasso Regression is a type of linear regression that adds a penalty term to the loss function to reduce model complexity and prevent overfitting. LASSO stands for Least Absolute Shrinkage and Selection Operator, and it works by penalizing the absolute values of regression coefficients. As the penalty increases, the model shrinks coefficient values, and some coefficients may become exactly zero. This property makes Lasso Regression especially useful for feature selection, as it can automatically remove less important features from the model.\n <\/p>\n\n <\/div>\n\n<\/div>\n\n\n\n
Machine learning models can sometimes memorise training data rather than learn real patterns. This leads to high variance and poor real-world performance.<\/p>\n\n\n\n
Regularisation is a way to control this problem by adding penalties to large coefficients.<\/p>\n\n\n\n
The main advantages of regularisation are as follows:<\/p>\n\n\n\n
Lasso Regression does this with L1 regularisation.<\/p>\n\n\n\n
Lasso Regression does this with L1 regularisation. If you want to understand regularisation concepts in more detail, you can also read this guide on Regularisation in Machine Learning<\/a>.<\/p>\n\n\n\n L1 regularisation adds the sum of the absolute values of coefficients to the loss function.<\/p>\n\n\n\n The mathematical objective function of Lasso Regression is as follows:<\/p>\n\n\n\n Loss = RSS + \u03bb \u2211 |\u03b2|<\/p>\n\n\n\n Where:<\/p>\n\n\n\n The amount of the penalty is controlled by the parameter lambda.<\/p>\n\n\n\n The coefficients shrink more aggressively as lambda increases.<\/p>\n\n\n\n Another unique property of L1 regularisation is that it can zero out coefficients. Some of the feature weights become exactly zero, removing those variables from the model.<\/p>\n\n\n\n This leads to a sparse model.<\/p>\n\n\n\n Curious about how these concepts work? Download HCL GUVI\u2019s<\/strong> free AI ebook<\/strong><\/a> to learn more about machine learning concepts, regression models, and real-world AI applications.\u00a0<\/p>\n\n\n\n In a sparse model, many of the feature coefficients are zero.<\/p>\n\n\n\n That is, the model only keeps the variables that are relevant and discards the irrelevant ones.<\/p>\n\n\n\n Sparse models are useful because they:<\/p>\n\n\n\n This is one of the biggest advantages of Lasso Regression over linear regression.<\/p>\n\n\n\n Feature selection is the identification of the most important variables in a dataset.<\/p>\n\n\n\n Traditional linear regression uses all features, even though some features have very little contribution.<\/p>\n\n\n\n It automatically drops the weak features by setting their coefficients to 0.<\/p>\n\n\n\n For example, a model to predict the price of a house might have features such as:<\/p>\n\n\n\n Some features, like wall colour or owner nickname, might not be very predictive.<\/p>\n\n\n\n Lasso Regression can set its coefficients to zero and remove them from the model.<\/p>\n\n\n\n Traditional linear regression uses all features, even though some features have very little contribution. To understand how regression models work before regularisation, you can also explore this Linear Regression Model in the Machine Learning Guide<\/a>.\u00a0<\/p>\n\n\n\n One of the most important parts of Lasso Regression is the lambda parameter.<\/p>\n\n\n\n It controls the degree of regularisation applied.<\/p>\n\n\n\n Different values of lambda give different results:<\/p>\n\n\n\n Choosing the right lambda is crucial to balancing bias and variance.<\/p>\n\n\n\n This is called the bias-variance tradeoff.<\/p>\n\n\n\n If the bias is low, the model may overfit.<\/p>\n\n\n\n A very high bias model could underfit.<\/p>\n\n\n\n Lasso Regression helps to find a happy medium.<\/p>\n\n\n\n Choosing the right lambda is crucial to balancing bias and variance. This concept is closely related to the Bias and Variance in Machine Learning Guide<\/a>.\u00a0<\/p>\n\n\n\n Lasso and Ridge Regression are both regularised regression techniques, but they behave differently.<\/p>\n\n\n\n If you have a lot of irrelevant features in your dataset, then Lasso Regression is often a better choice.<\/p>\n\n\n\nL1 Regularisation Explained<\/strong><\/h2>\n\n\n\n
\n
Sparse Model is defined as:<\/strong><\/h2>\n\n\n\n
\n
How Lasso Regression Does Feature Selection<\/strong><\/h2>\n\n\n\n
\n
Meaning of the lambda parameter<\/strong><\/h2>\n\n\n\n
\n
Ridge Regression vs Lasso Regression<\/strong><\/h2>\n\n\n\n
Lasso Regression:<\/strong><\/h3>\n\n\n\n
\n
Ridge regression (RR)<\/strong><\/h3>\n\n\n\n
\n