Bayes Decision Rule

For each sample x, if h can minimize the conditional risk R(h(x)|x), the overall risk will also be minimized. This gives rise to the Bayes decision rule: to minimize the overall risk, we only need to select the category label that minimizes the conditional risk R(c|x) for each sample, that is,

At this time, h∗ is called the Bayes optimal classifier, and the corresponding overall risk R(h∗) is called the Bayes risk.