The open source ecosystem for AutoML

By reviewing the history of automated ML, it is evident that, in the early days, the focus had always been on hyperparameter optimization. The earlier tools, such as AutoWeka and HyperoptSkLearn, and later TPOT, had an original focus on using Bayesian optimization techniques to find the most suitable hyperparameters for the model. However, this trend shifted left to include model selection, which eventually engulfed the entire pipeline by including feature selection, preprocessing, construction, and data cleaning. The following table shows some of the prominent automated ML tools that are available, including TPOT, AutoKeras, auto-sklearn, and Featuretools, along with their optimization techniques, ML tasks, and training frameworks:

Figure 3.1 – Features of automated ML frameworks

For several of the examples in this chapter, we will be using the MNIST database of handwritten digits. We will be using the scikit-learn...