Residual Mapping

Residual mapping is the corresponding relationship based on which the residual network is constructed. Its common form is H(x) = F(x) + x, where F(x) is the residual function.

Related definitions

In mathematical statistics, residuals represent the difference between actual observed values and fitted values, and contain important information about the model.

Assume that the hidden mapping that needs to be learned between layers is H(x), and the residual mapping is F(x) = H(x) – x. Then the mapping H(x) that originally needs to be learned is the residual function F(x) + x, that is, the residual is defined as: residual = output – input.

Residual Mapping and Networks

Residual mapping is achieved by adding an identity mapping, that is, introducing a shortcut connection between the output and the input at the same time, rather than simply stacking the network.

The original function H(x) to be learned is converted into F(x) + x. This not only solves the problem of gradient disappearance in the network, but also makes the network very deep, thereby constructing the residual network ResNet.