Introducing the cross-entropy loss
The binary CEL, also called the log loss, is often used as the cost function in logistic regression. This is the loss that the logistic regression model will attempt to minimize by moving the parameters. This function takes the predicted probabilities and the corresponding targets as the input and outputs a scalar score, indicating the goodness of fit. For a single observation with a target of y i and predicted probability of p i, the loss is calculated as follows:
Q i(y i, p i) = − [ y i logp i + (1 − y i)log(1 − p i)]
Summing up all individual losses gives the total binary CEL:
Q(y, p) =  1 _ N ∑ i N Q i =  1 _ N ∑ i=1 N − [ y i logp i + (1 − y i)log(1 − p i)]
The binary CEL function is a suitable choice for binary classification problems...
