Linearity versus non-linearity

Another consideration is decision boundaries. Some algorithms, such as logistic regression or Support Vector Machine (SVM), can learn linear decision boundaries while others, such as tree-based algorithms, can learn non-linear decision boundaries. While linear decision boundaries are relatively easy to calculate and interpret, you should be aware of errors that linear algorithms will generate in the presence of non-linear relationships.

Drawing decision boundaries

The following code snippet will allow you to examine the decision boundaries of different types of algorithms:

import matplotlib.cm as cm

# This function will scale training datatset and train given classifier.
# Based on predictions...