Summary
When developing machine learning models, one of the main goals is for the model to be capable of generalizing so that it can be applicable to future unseen data, instead of just learning a set of instances very well but performing poorly on new data. Accordingly, a methodology for validation and testing was explained in this chapter, which involved splitting the data into three sets: a training set, a dev set, and a test set. This approach eliminates the risk of bias. After this, the chapter covered how to evaluate the performance of a model for both classification and regression problems. Finally, we covered how to analyze the performance and perform error analysis for each of the sets and detect the condition affecting the model's performance.
In the next chapter, we will focus on applying different algorithms to a real-life dataset, with the underlying objective of applying the steps learned here to choose the best performing algorithm for the case study.
