Representer Theorem

The representation theorem is a theorem in statistical learning that shows that the minimum of the regularized risk function on a reproducing kernel Hilbert space can be represented as a linear combination of kernel functions.

Practical application examples

On the L2 regularization problem:

The representation theorem states that for any L2 regularized problem, the optimal w* can be obtained by a linear combination of βn and Zn.

Theoremssignificance

• Simplified the regularized empirical risk minimization problem;
• The infinite-dimensional minimization problem is reduced to a three-dimensional vector of search for optimal coefficients, which can then be solved by standard function minimization algorithms;
• Provide a theoretical basis for generalizing general machine learning problems to implementable algorithms.

Related terms: linear combination, L2 regularization