Avoiding overfitting in the model
The fitting of the training data causes the model to determine the weights and biases along with the activation function values. When the algorithm does too well in some training dataset, it is said to be too much aligned to that particular dataset. This leads to high variance in the output values when the test data is very different from the training data. This high estimate variance is calledoverfitting. The predictions are affected due to the training data provided.
There are many possible ways to handle overfitting in neural networks. The first is regularization, similar to regression. There are two kinds of regularizations:
- L1 or lasso regularization
- L2 or ridge regularization
- Max norm constraints
- Dropouts in neural networks
Regularization introduces a cost term to impact the activation function. It tries to change most of the coefficients by bringing in more features with the objective function. Hence, it tries to push the coefficients for many variables...
