Individual Learner

Individual LearnerIt is a relative concept, which refers to the learner before integration in ensemble learning.

According to the way individual learners are generated, ensemble learning methods can be divided into the following two categories:

• There are strong dependencies and serialization methods must be generated serially, such as Boosting.
• There is no strong dependency between parallel methods that can be generated simultaneously. Representative examples are Bagging and Random Forest.

Boosting is an algorithm that can promote weak learners to strong learners. It first trains a base learner from the initial training set, then adjusts the distribution of training samples based on the base learner so that the training samples that the base learner got wrong receive more attention in the future, and then trains the next base learner based on the adjusted sample distribution; this process is repeated until the number of base learners reaches a predetermined value T, and finally these T learners are weightedly combined.

Bagging is a representative of parallel ensemble learning methods, which is based on bootstrap sampling and focuses on reducing variance. The random forest algorithm is an extended variant of bagging, which builds a bagging ensemble based on decision trees and further introduces random attribute selection in the training process of decision trees.

References

【1】"Machine Learning" Notes - Ensemble Learning (Zhihu Article)