Introduction to non-parametric models and decision trees
In this section, we're going to formally define what non-parametric learning algorithms are, and introduce some of the concepts and math behind our first algorithm, called decision trees.
Non-parametric learning
Non-parametric models do not learn parameters. They do learn characteristics or attributes about the data, but not parameters in the formal sense. We will not end up extracting a vector of coefficients. The easiest example is a decision tree. A decision tree is going to learn where to recursively split data so that its leaves are as pure as possible. So, in that sense, the decision function is a splitting point for each leaf that is not a parameter.
Characteristics of non-parametric learning algorithms
Non-parametric models tend to be a bit more flexible and do not make as many assumptions about the underlying structure of the data. Many linear models, or parametric models, for instance, assume that a normal distribution for each...
