Regression is one of the oldest and yet quite powerful tools for mathematical modelling, classification, and prediction. Regression finds application in varied fields ranging from engineering, physical science, biology, and the financial market to social sciences. It is the basic tool in the hand of a data scientist.

Regression is normally the first algorithm that people in machine learning work with. It allows us to make predictions from data by learning the relationship between the dependent and independent variables. For example, in the case of house price estimation, we determine the relationship between the area of the house (independent variable) and its price (dependent variable); this relationship can be then used to predict the price of any house given its area. We can have multiple independent variables impacting the dependent variable. Thus, there are two...

As discussed earlier, in regression we define the loss function or objective function and the aim is to find the coefficients such that the loss is minimized. In this recipe, you will learn how to define loss functions in TensorFlow and choose a proper loss function depending on the problem at hand.

Declaring a loss function requires defining the coefficients as variables and the dataset as placeholders. One can have a constant learning rate or changing learning rate and regularization constant. In the following code, let m be the number of samples, n the number of features, and P the number of classes. We should define these global parameters before the code:

m = 1000
n = 15
P = 2

From your high school math, you must know that a function's first derivative is zero at its maxima and minima. The gradient descent algorithm is based on the same principle--the coefficients (weights and biases) are adjusted such that the gradient of the loss function decreases. In regression, we use gradient descent to optimize the loss function and obtain coefficients. In this recipe, you will learn how to use the gradient descent optimizer of TensorFlow and some of its variants.

The update of coefficients (W and b) is done proportionally to the negative of the gradient of the loss function. There are three variations of gradient descent depending on the size of the training sample...

Most of you will already be familiar with Pandas and its usefulness in handling large dataset files. TensorFlow also offers methods to read from the files. In the first chapter, we went through the recipe for reading from files in TensorFlow; in this recipe, we will focus on how to read from CSV files and preprocess the data before training.

We will consider the Boston housing price dataset (http://lib.stat.cmu.edu/datasets/boston) collected by Harrison and Rubinfield in 1978. The dataset contains 506 sample cases. Each house is assigned 14 attributes:

• CRIM: per capita crime rate by town
• ZN: Proportion of residential land zoned for lots over 25,000 sq.ft
• INDUS: Proportion...

In this recipe, we will perform simple linear regression based on the number of rooms (RM) on the Boston house price dataset.

Our goal is to predict the house price given in the last column (MEDV). In this recipe, we load the dataset from the TensorFlow Contrib dataset directly. We optimize the coefficients for an individual training sample using the stochastic gradient descent optimizer.

We proceed with the recipe as follows:

1. The first step is to import all the packages that we will need:

We can do multiple linear regression on the same data by making a slight modification in the declaration of weights and placeholders. In the case of multiple linear regression, as each feature has different value ranges, normalization becomes essential. Here is the code for multiple linear regression on the Boston house price dataset, using all the 13 input features.

Here is how we proceed with the recipe:

1. The first step is to import all the packages that we will need:

import tensorflow as tf
 import numpy as np
 import matplotlib.pyplot as plt

2. We need to normalize the feature data since the data ranges of all the features are varied. We define a normalize...

This recipe is based on the logistic regressor for MNIST provided at https://www.tensorflow.org/get_started/mnist/beginners, but we will add some TensorBoard summaries to understand it better. Most of you must already be familiar with the MNIST dataset--it is like the ABC of machine learning. It contains images of handwritten digits and a label for each image, saying which digit it is.

For logistic regression, we use one-hot encoding for the output Y. Thus, we have 10 bits representing the output; each bit can have a value either 0 or 1, and being one-hot means that for each image in label Y, only one bit out of the 10 will have value 1, the rest will be zeros. Here, you can see the image of the handwritten numeral 8, along with its hot encoded value [0 0 0 0 0 0 0 0 1 0]: