Working with lasso regression
Lasso regression is another type of regularized linear regression. It is similar to ridge regression but differs in terms of the specific process of calculating the magnitude of the coefficients. Specifically, it uses the L1 norm of the coefficients, which consists of the total sum of absolute values of the coefficients, as the penalty that’s added to the OLS loss function.
The lasso regression cost function can be written as follows:
L lasso = RSS + λ∑ j=1 p | β j|
The key characteristic of lasso regression is that it can reduce some coefficients exactly to 0, effectively performing variable selection. This is a consequence of the L1 penalty term and is not the case for ridge regression, which can only shrink coefficients close to 0. Therefore, lasso regression is particularly useful when we believe that only a subset of the predictors matters when it comes to predicting the outcome.
In addition...
