Before moving on to the core topic, we would like to build a foundation for getting there. Hence, this segment of the chapter is very important. It might look familiar to you and many of you will be cognizant about this. However, going through this channel will set the flow.

A vector is an object that has both a direction and magnitude. It is represented by an arrow and with a coordinate (x, y) in space, as shown in the following plot:

As shown in the preceding diagram, the vector OA has the coordinates (4,3): 

Vector OA= (4,3)

However, it is not sufficient to define a vector just by coordinates—we also need a direction. That means the direction from the x axis.

Linear separability implies that if there are two classes then there will be a point, line, plane, or hyperplane that splits the input features in such a way that all points of one class are in one-half space and the second class is in the other half-space.

For example, here is a case of selling a house based on area and price. We have got a number of data points for that along with the class, which is house Sold/Not Sold:

In the preceding figure, all the N, are the class (event) of Not Sold, which has been derived based on the Price and Area of the house and all the instances of S represent the class of the house getting sold. The number of N and S represent the data points on which the class has been determined.

In the first diagram, N and S are quite close and happen to be more random, hence, it's difficult to have linear separability...

Many of you will have guessed it right. We use hyperplanes when it comes to more than 3D. We will define it using a bit of mathematics.

A linear equation looks like this: y = ax + b has got two variables, x and y, and a y-intercept, which is b. If we rename y as x₂ and x as x₁, the equation comes out as x₂=ax₁ + b which implies ax₁ - x₂ + b=0. If we define 2D vectors as x= (x₁,x₂) and w=(a,-1) and if we make use of the dot product, then the equation becomes w.x + b = 0. 

So, a hyperplane is a set of points that satisfies the preceding equation. But how do we classify with the help of hyperplane?

We define a hypothesis function h:

h(x_i) = +1 if w.x_i + b ≥ 0

-1 if w.x_i + b < 0

This could be equivalent to the following:

h(x_i)= sign(w.x_i + b) 

It could also be equivalent to the following...

Now we are ready to understand SVMs. SVM is an algorithm that enables us to make use of it for both classification and regression. Given a set of examples, it builds a model to assign a group of observations into one category and others into a second category. It is a non-probabilistic linear classifier. Training data being linearly separable is the key here. All the observations or training data are a representation of vectors that are mapped into a space and SVM tries to classify them by using a margin that has to be as wide as possible:

Let's say there are two classes A and B as in the preceding screenshot.

And from the preceding section, we have learned the following:

g(x) = w. x + b

Where:

• w: Weight vector that decides the orientation of the hyperplane
• b: Bias term that decides the position of the hyperplane in n-dimensional space by biasing it

The...

We have already seen that SVM works smoothly when it comes to having linear separable data. Just have a look at the following figure; it depicts that vectors are not linearly separable, but the noticeable part is that it is not being separable in 2D space:

With a few adjustments, we can still make use of SVM here.

Transformation of a two-dimensional vector into a 3D vector or any other higher dimensional vector can set things right for us. The next step would be to train the SVM using a higher dimensional vector. But the question arises of how high in dimension we should go to transform the vector. What this means is if the transformation has to be a two-dimensional vector, or 3D or 4D or more. It actually depends on the which brings separability into the dataset.

We're going to explain the types of in this section.

Let's say there are two vectors, x₁ and x₂, so the linear kernel can be defined by the following:

K(x_1, x₂)= x_{1 .} x₂

If there are two vectors, x₁ and x₂, the linear kernel can be defined by the following:

K(x_1, x₂)= (x_{1 .} x₂ + c)^d

Where:

• c: Constant
• d: Degree of polynomial:

def polynomial_kernel(x1, x2, degree, constant=0): 
    result = sum([x1[i] * x2[i] for i in range(len(x1))]) + constant 
    return...

Here, we are taking a breast cancer dataset wherein we have classified according to whether the cancer is benign/malignant.

The following is for importing all the required libraries:

import pandas as pd
import numpy as np
from sklearn import svm, datasets
from sklearn.svm import SVC
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
from sklearn.utils import shuffle
%matplotlib inline

Now, let's load the breast cancer dataset:

BC_Data = datasets.load_breast_cancer()

The following allows us to check the details of the dataset:

print(BC_Data.DESCR)

This if for splitting the dataset into train and test:

X_train, X_test, y_train, y_test = train_test_split(BC_Data.data, BC_Data.target, random_state...