Machine learning basics

ML is a concept that describes the process of a set of generic algorithms analyzing your data, and providing you with interesting data without writing any specific codes for your problem. 

Alternatively, we can look at ML as a black box how cutting edge scientists are using it to do something crazy like detecting epilepsy or cancer disease, yet your simple email inbox is using it to filter spam every day. 

On a larger level, ML can be classified into the following two categories:

• Supervised learning
• Unsupervised learning

Supervised learning

With supervised learning, your main task is to develop a function that maps inputs to outputs. For example, if you have input variables (x) and an output variable (y), then you can use an algorithm to learn the mapping function from the input to the output:

y = f(x)

The goal is to approximate the mapping function so well that when you have new input data (x), you can predict the output variables (y) for it.

For example, you have a bunch of fruits and baskets. You have started labeling the fruits and baskets as apple, banana, strawberry, etc., When you are done with labeling all the fruits into their corresponding baskets, now your job is to label the new fruit that comes in. You have already learnt all the fruits and their details by labeling them. Based on the previous experience you can now label the new fruit based on its attributes like color, size and pattern.

Unsupervised learning

In this case, you only have input data (x) and no corresponding output variables. The goal for unsupervised learning is to model the underlying structure or distribution in the data in order to learn more about it.

In unsupervised learning, you may not have any data in the beginning. Say for example on the same scenario discussed above in supervised learning, you have a basket full of fruits and you are asked to group them into similar groups. But you don't have any previous data or there are no training or labeling is done earlier. In that case, you need to understand the domain first because you have no idea whether the input is a fruit or not. In that case, you need to first understand all the characteristics of every input and then to try to match with every new input. May be at the final step you might have classified all the red color fruits into one baskets and the green color fruits into another basket. But not an accurate classification. This is called as unsupervised learning. 

Linear regression - supervised learning

Let's look at one simple example of linear regression with its implementation on TensorFlow. 

Let's predict the price of a house, looking at the prices of other houses in the same area along with its size information:

We have a house that costs $82,000 and another one at $55,000. Now, our task is to find the price of the third house. We know the sizes of all of the houses along with the prices, and we can map this data in a graph. Let's find out the price of the third house based on the other two data points that we have:

You may wonder how to draw the line now. Draw a random line close to all the points marked on the graph. Now, calculate the distance from each point to the line and add them together. The result of this is the error value. Our algorithm should move toward minimizing the error, because the best fit line has a lower error value. This procedure is known as a gradient descent. 

The prices of all the houses in the particular locality are mapped on the graph accordingly. Now, let's plot the values of the two houses that we already know:

After that, let's draw one line that passes closer to most of the values. The line fits the data perfectly. From this, we should be able to identify the price of house number three:

Based on the size of the data provided for home number three, we can map the size of the data provided for the home number on the graph. This will allow us to figure out the connecting point on the line drawn through all points. That maps to $98,300 on the y axis. This is known as linear regression:

Let's try to translate our problem into the form of a pseudocode:

def estimate_house_price(sqft, location): 
 price = 0 
 #In my area, the average house costs 2000 per sq.ft 
 price_per_sqft = 2000 
 if location == "vegas": 
     #but some areas cost a bit more 
     price_per_sqft = 5000 
 elif location == "newyork": 
     #and some areas cost less 
     price_per_sqft = 4000 
 #start with a base price estimate based on how big the place is 
 price = price_per_sqft * sqft 
 return price

This will be your typical method for estimating the house price. We can keep adding more and more condition checks to this, but it can't be controlled beyond the point at which the number of locations or the parameter increases. For a typical house price estimation, there are a lot of other factors also considered, such as the number of rooms, locations in the vicinity, schools, gas stations, hospitals, water level, transport, and so on. We can generalize this function into something very simple and guess the right answer:

def estimate_house_price(sqft, location): 
 price = < DO MAGIC HERE >
 return price

How can we identify a line that fits perfectly without writing condition checks? Typically, the linear regression line is represented in the following form: 

Y = XW +b

In our example, let's put this into a more simple form in order to gain a better understanding:

prediction = X * Weight +bias

Weight is the slope of the line and bias is the intercept (the value of Y when X = 0). After constructing the linear model, we need to identify the gradient descent. The cost function identifies the mean squared error to bring out the gradient descent:

Let's represent the cost function through pseudocode to solve our problem of estimating the house price:

def estimate_house_price(sqft, location):
 price = 0
 #and this
 price += sqft * 235.43
 #maybe this too
 price += location * 643.34
 #adding a little bit of salt for a perfect result 
 price += 191.23
 return price

The values 235.43, 643.34, and 191.23 may look random, but with these values we are able to discover the estimation of any new house. How do we arrive at this value? We should do an iteration to arrive at the right value while reducing our error in the right direction:

def estimate_house_price(sqft, location):
 price = 0
 #and this
 price += sqft * 1.0
 #maybe this too
 price += location * 1.0
 #adding a little bit of salt for a perfect result
 price += 1.0
 return price

So, we will start our iterations from 1.0 and start minimizing errors in the right direction. Let's put this into code using TensorFlow. We will look at the methods that are used in more depth later on:

#import all the necessary libraries
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy

#Random number generator
randnumgen = numpy.random

#The values that we have plotted on the graph
values_X = 
  numpy.asarray([1,2,3,4,5.5,6.75,7.2,8,3.5,4.65,5,1.5,4.32,1.65,6.08])
values_Y = 
 numpy.asarray([50,60,65,78,89,104,111,122,71,85,79,56,81.8,55.5,98.3])

# Parameters
learning_rate = 0.01
training_steps = 1000
iterations = values_X.shape[0]

# tf float points - graph inputs
X = tf.placeholder("float")
Y = tf.placeholder("float")

# Set the weight and bias
W = tf.Variable(randnumgen.randn(), name="weight")
b = tf.Variable(randnumgen.randn(), name="bias")

# Linear model construction
# y = xw + b
prediction = tf.add(tf.multiply(X, W), b)

#The cost method helps to minimize error for gradient descent. 
#This is called mean squared error.
cost = tf.reduce_sum(tf.pow(prediction-Y, 2))/(2*iterations)

# In TensorFlow, minimize() method knows how to optimize the values for # weight & bias. 
optimizer = 
   tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

#assigning default values 
init = tf.global_variables_initializer()

#We can start the training now
with tf.Session() as sess:

    # Run the initializer. We will see more in detail with later       
    #chapters
     sess.run(init)

    # Fit all training data
     for step in range(training_steps):
         for (x, y) in zip(values_X, values_Y):
             sess.run(optimizer, feed_dict={X: x, Y: y})
             c = sess.run(cost, feed_dict={X: values_X, Y:values_Y})
             print("Step:", '%04d' % (step+1), "cost=", "  
                             {:.4f}".format(c), \
                             "W=", sess.run(W), "b=", sess.run(b))

     print("Successfully completed!")
     # with this we can identify the values of Weight & bias
     training_cost = sess.run(cost, feed_dict={X: values_X, Y: 
                                               values_Y})
     print("Training cost=", training_cost, "Weight=", sess.run(W), 
           "bias=", sess.run(b))

     # Lets plot all the values on the graph
     plt.plot(values_X, values_Y, 'ro', label='house price points')
     plt.plot(values_X, sess.run(W) * values_X + sess.run(b), 
                                       label='Line Fitting')
     plt.legend()
     plt.show()

You can find the same code from our GitHub repository (https://github.com/PacktPublishing/Machine-Learning-Projects-for-Mobile-Applications) under Chapter01.