Occam's Razor

Occam's razorIt means that if there are multiple hypotheses that are consistent with observations, the simplest one is chosen. Occam's razor is often used as a heuristic technique. It is a tool to help people develop theoretical models and cannot be used as a basis for judging theories.

The origin of Occam's razor

Occam's razor is also called "Occam's razor" in some places. Its Latin name is lex parsimoniae, which means the law of simplicity.

It is a problem-solving rule proposed by William of Ockham, a 14th-century logician and Franciscan monk, who said in his Commentary on the Book of Proverbs, Volume 2, Question 15, "Never waste more, and do what you can do with less."

In other words, if there are many theories about the same problem, and each of them makes equally good predictions, then the one that makes the fewest assumptions should be chosen. Although more complex methods usually make better predictions, as long as predictive power is not a concern (i.e., the results are roughly the same), the fewer assumptions, the better.

Application of Occam's razor

The prototype of Occam's razor applies only to models with equal explanatory power (that is, it simply tells you to choose the simpler model among equally good ones).

A more general form of Occam's razor can be derived from Bayesian model comparison.

It is based on Bayes factors and can be used to compare models that do not fit the observations equally well. These models sometimes find the best balance between explanatory power and complexity.

In general, the exact value of the Bayes factor is difficult to obtain, but there are many methods to give an approximate value, such as Akaike's information criterion, Bayesian information criterion, variational Bayes method, false discovery rate and Laplace method. Many artificial intelligence researchers use these methods in Occam learning.