Understanding Logistic Regression

Logistic regression is one of the most widely used classification methods, and it works well when data is linearly separable. The objective of logistic regression is to squash the output of linear regression to classes 0 and 1.

Revisiting Linear Regression

In the case of linear regression, our function would be as follows:

Figure 7.2: Equation of linear regression

Here, x refers to the input data, y is the target variable, and θ0 and θ1 are parameters that are learned from the training data.

Also, the cost function in case of linear regression, which is to be minimized is as follows:

Figure 7.3: Linear regression cost function

This works well for continuous data, but the problem arises when we have a target variable that is categorical, such as, 0 or 1. When we try to use linear regression to predict the target variable, we can get a value anywhere between −∞ to +∞, which is not what we need.

Logistic Regression

If a response variable has binary values, the assumptions...