Feature selection

The are many factors that influence the success or failure of a model, such as sampling, data quality, feature creation, and model selection, several of which we have not covered. One of those critical factors is feature selection. Feature selection is simply the process of choosing or systematically determining the best features for a model from an existing set of features. We have done some simple feature selection already. In the previous section, we removed features that had high VIFs. In this section, we will look at some methods for feature selection. The methods presented in this section fall into two categories: statistical methods for feature selection and performance-based methods for feature selection. Let’s start with statistical methods.

Statistical methods for feature selection

Statistical methods for feature selection rely on the primary tool that we have used throughout the previous chapters: statistical significance. The methods presented...