Exploring the gradient descent technique
The goal of any simulation algorithm is to reduce the difference between the values predicted by the model and the actual values returned by the data. This is because a lower error between the actual and expected values indicates that the algorithm has done a good simulation job. Reducing this difference simply means minimizing the objective function that the model being built is based on.
Defining descent methods
Descent methods are iterative methods that, starting from an initial point x0 ∈ Rn, generate a sequence of points {xn} n ∈ N defined by the following equation:
Here, the 
vector is a search direction and the 
scalar is a positive parameter called step length, which indicates the distance by which we move in the 
direction.
In a descent method, the 
vector and the 
parameter are chosen to guarantee the decrease of the objective f function at each iteration, as follows:
