Margin Theory
Interval TheoryIt is a concept in support vector machines, where the interval refers to the minimum distance between two types of samples divided by a hyperplane, and the interval theory can be used to explain that when the training error of the AdaBoost algorithm is 0, continuing training can further improve the generalization performance of the model.
Let x and y represent the input and output spaces of the samples, D be the true distribution of the samples on x · y, and S= is a sampling on the sample D. In the hypothesis space H, the base classifiers h : x → y are weighted combined to form an ensemble classifier. f ∈ C(H), which is the convex hull of H.
In the AdaBoost algorithm, the ensemble classifier f(x) is produced by weighted voting of a series of base classifiers, that is, . Where
, based on the strong classifier definition, the following interval can be defined:
That is, the weighted difference between correct votes and incorrect votes.