In this work, we study the continual semantic segmentation problem, where thedeep neural networks are required to incorporate new classes continuallywithout catastrophic forgetting. We propose to use a structuralre-parameterization mechanism, named representation compensation (RC) module,to decouple the representation learning of both old and new knowledge. The RCmodule consists of two dynamically evolved branches with one frozen and onetrainable. Besides, we design a pooled cube knowledge distillation strategy onboth spatial and channel dimensions to further enhance the plasticity andstability of the model. We conduct experiments on two challenging continualsemantic segmentation scenarios, continual class segmentation and continualdomain segmentation. Without any extra computational overhead and parametersduring inference, our method outperforms state-of-the-art performance. The codeis available at \url{https://github.com/zhangchbin/RCIL}.