Introducing simple linear regression

Linear regression is a kind of steampunk machine learning. It was invented in the time of Sherlock Holmes, long before the first electronic computer was invented and the term machine learning was coined. The term regression and its calculation algorithm was introduced by the English polymath Sir Francis Galton in 1886, in the publication named Regression towards Mediocrity in Hereditary Stature. Galton proposed the concept while performing research on how to create the perfect breed of people. The task of regression emerged from the need to predict the child's body parameters given the parent's body measurements. So nowadays, Sir Galton is mostly remembered as the father of eugenics rather than as an inventor of the first machine learning algorithm. Later in this chapter, we will follow the footsteps of Galton (but not too far), and...