Conditional Computation

Conditional computation is a technique to reduce the total amount of computation by performing computation only when it is needed.There are currently a variety of solutions for when and how much computation is needed. However, these algorithms are not necessarily applicable to existing hardware because they tend to introduce dynamic computation graphs, while existing hardware prefers to use static computation graphs.