So, now we know some ways to measure how well our model is performing. In fact, if we wanted to, we could create a super sophisticated, complicated model that could predict every observation without error. For example, we could create a model that would take the index of the row of the observation and return the exact answer for each of those rows. It might be a really big function with a lot of parameters, but it would return the correct answers.
So, what's the problem with this? Well, the problem is that it would not generalize to new data. Our complicated model would predict really well for the data that we would expose it to, but once we try some new input data (that isn't part of our training dataset), the model would likely perform poorly.
We call this type of model (that doesn't generalize) a model that has been overfit. That is, our process of...
