Ensembling

Imagine that a group of friends are deciding which movie they want to see together. For this, they select their movie of choice from a set of, say, five or six movies. At the end, all their votes are collected and read. The movie with the maximum votes is picked and watched. What just happened is a real-life example of the ensembling approach. Basically, multiple entities act on a problem and give their selection out of a collection of discrete choices (in the case of a classification problem). The selection that was suggested by the maximum number of entities is chosen as the predicted choice.

This explanation was a general approach to ensembling. From the perspective of machine learning, it just means that multiple machine learning programs act on a problem that can be either of type classification or regression. The output from each machine learning algorithm is collected. The results from all the algorithms are then analyzed with different approaches like voting, averaging...