Variational Inference

Variational inference is a method for approximate inference in probabilistic graphical models. Compared with randomization methods based on sampling, it is a deterministic approximation method.

definition

The key points of the idea of variational inference can be summarized as follows:

• Use a known simple distribution to approximate a complex distribution that needs to be inferred;
• Restrict the types of approximate distributions;
• An approximate posterior distribution with a local optimum but a definite solution is obtained.

The original goal is to infer the required distribution p based on the existing data; when p is not easy to express and cannot be solved directly, you can try to use variational inference, that is, to find a distribution q that is easy to express and solve. When the difference between q and p is very small (the KL divergence distance is the smallest), q can be used as an approximate distribution of p and become the output result.

application

Variational inference is often used in Bayesian estimation and machine learning to approximate complex integrals, and is suitable for inference of various complex models.

References 【1】http://crescentmoon.info/2013/10/03/ Variational Inference Learning Notes 1——Concept Introduction