Generalization Error Bound
The upper bound of the generalization error refers to the maximum value allowed for the generalization error. Exceeding this upper bound will affect the feasibility of machine learning.
Generalization error refers to the error generated in the process of generalizing from the training set to outside the training set. It is usually obtained by subtracting the training error from the error outside the training set, that is, the error expectation on the entire input space.
Because the upper bound of the error is widely general, it is enlarged many times during the derivation process, and the upper bound finally obtained is very loose. The significance of practical application lies mainly in its relative value rather than its absolute value.
There are two factors that affect generalization error: data volume and model complexity. Generally, the amount of data is increased as much as possible, while complexity needs to be considered comprehensively. In order to obtain the best performance, a balance needs to be achieved between the two.