Understanding the perceptron

In the 1950s, American psychologist and artificial intelligence researcher Frank Rosenblatt invented an algorithm that would automatically learn the optimal weight coefficients w₀ and w₁ needed to perform an accurate binary classification: the perceptron learning rule.

Rosenblatt's original perceptron algorithm can be summed up as follows:

1. Initialize the weights to zero or some small random numbers.
2. For each training sample s_i, perform the following steps:
   1. Compute the predicted target value ŷ_i.
   2. Compare ŷ_i to the ground truth y_i, and update the weights accordingly:
      • If the two are the same (correct prediction), skip ahead.
      • If the two are different (wrong prediction), push the weight coefficients w₀ and w₁ towards the positive or negative target class respectively.

Let's have a closer look at the last step, which is the weight...