What is an ensemble?

An ensemble is a collection of multiple models which are combined to create a stronger model with better predictive performance. An ensemble of models built on samples of the data can become a powerful predictor by averaging the errors of each individual model.

Generally, ensembles perform better than a single model because they are less sensitive to outliers in your training data. A key drawback of Machine Learning is that the algorithm has the potential to overfit your data, so that its performance on your training data is very good, but it does not generalize well to new data. Ensembles can learn multiple models over different subsamples of data and take a majority vote to predict results to mitigate the risk of overfitting a single model on all the data.