Some mathematical tools
Before introducing backpropagation, we need to review some mathematical tools from calculus. Don’t worry too much; we’ll briefly review a few areas, all of which are commonly covered in high school-level mathematics.
Vectors
We will review two basic concepts of geometry and algebra that are quite useful for machine learning: vectors and the cosine of angles. We start by giving an explanation of vectors. Fundamentally, a vector is a list of numbers. Given a vector, we can interpret it as a direction in space. Mathematicians most often write vectors as either a column x or row vector xT. Given two column vectors u and v, we can form their dot product by computing 
. It can be easily proven that 
where 
is the angle between the two vectors.
Here are two easy questions for you. What is the result when the two vectors are very close? And what is the result when the two vectors are the same?
