Information Plane

The Information Plane (IP) is a two-dimensional representation of the information trajectory of hidden layers during the training process of deep neural networks. The coordinate axes are the mutual information between the input and the hidden layer, and the mutual information between the output and the hidden layer. By analyzing the IP, one can uncover the internal information processing mechanisms of the network, optimize model architecture and training strategies, and enhance model performance and generalization capabilities. The application value of the IP lies in providing a theoretical tool that helps researchers gain a deeper understanding of the learning process and information transfer characteristics of deep learning models.