Modern convolutional networks are not shift-invariant, as small input shiftsor translations can cause drastic changes in the output. Commonly useddownsampling methods, such as max-pooling, strided-convolution, andaverage-pooling, ignore the sampling theorem. The well-known signal processingfix is anti-aliasing by low-pass filtering before downsampling. However, simplyinserting this module into deep networks degrades performance; as a result, itis seldomly used today. We show that when integrated correctly, it iscompatible with existing architectural components, such as max-pooling andstrided-convolution. We observe \textit{increased accuracy} in ImageNetclassification, across several commonly-used architectures, such as ResNet,DenseNet, and MobileNet, indicating effective regularization. Furthermore, weobserve \textit{better generalization}, in terms of stability and robustness toinput corruptions. Our results demonstrate that this classical signalprocessing technique has been undeservingly overlooked in modern deep networks.Code and anti-aliased versions of popular networks are available athttps://richzhang.github.io/antialiased-cnns/ .