Summary
This chapter has been a shorter one and largely a visual one. The only heavy math came in the proof of the bias-variance decomposition of the generalization error. However, the visual approach has been useful in explaining concepts of overfitting, underfitting, and generalization. At a superficial level, these concepts are intuitive and need very little explanation. You will have probably encountered them before. However, a more thorough understanding of these concepts is crucial if we’re not to be misled by them when we’re building predictive models. That thorough understanding has required us to learn additional concepts. Across the whole chapter, the concepts we have learned about included the following:
- Model complexity and how we broadly think of this as being related to the number of parameters in a model
- Overfitting to the noise in a dataset and how it increases as we increase the complexity of a model
- Underfitting to the general trends...
