Summary

This chapter has focused on a single, but important, concept – loss functions. Loss functions are important because they help us measure how good our predictive models are and, more generally, how well one mathematical object approximates another. They are also important because we can minimize them with respect to our model parameters, and so we can use loss functions, or more specifically risk functions, to fit our models to training data. In this chapter, we have learned about the different aspects of risk functions and how to minimize them. Specifically, we have learned about the following:

• What a loss function is and what it measures
• That a risk function is the expectation value of a loss function
• What the empirical risk function is and how it is calculated from training data
• How least squares minimization is a form of empirical risk minimization and can be used to estimate optimal parameter values for a model
• How OLS regression performs...