Backpropagation Through Time

Backpropagation through time is a backpropagation algorithm applied to recurrent neural networks (RNNs).

BPTT can be viewed as the standard back-propagation algorithm applied to RNNs, where each time step represents a computational layer and its parameters are shared across computational layers.

Because the RNN shares the same parameters at all time steps, the error at one time step must be back-propagated "through time" to all previous time steps, hence the name of the algorithm. When processing long sequences (hundreds of inputs), a truncated version of BPTT is often used to reduce computational cost. The truncated BPTT stops back-propagating errors after a fixed number of steps.