Validating the model
After you train your model, you will need to determine whether it will perform well in predicting the target on future new data, and that is the validation process: you must validate the model performance on a labeled dataset that was not used in training – the validation dataset that was built during the dataset splitting phase.
Model validation
Recall what we have discussed in Chapter 3: in the ML problem framing phase, you define the business problem and craft a business metric to measure model success. Now, in this model validation phase, the model validation metric needs to be linked to that business metric as closely as possible.
Earlier in this chapter, we have defined the cost function, which is used to find the optimized model. The cost function is also used for ML model validation. For regression problems, the cost function (the gap between the model value and the actual value) is usually the MSE, which was discussed in the previous...
