Despite advances in feature representation, leveraging geometric relations iscrucial for establishing reliable visual correspondences under large variationsof images. In this work we introduce a Hough transform perspective onconvolutional matching and propose an effective geometric matching algorithm,dubbed Convolutional Hough Matching (CHM). The method distributes similaritiesof candidate matches over a geometric transformation space and evaluate them ina convolutional manner. We cast it into a trainable neural layer with asemi-isotropic high-dimensional kernel, which learns non-rigid matching with asmall number of interpretable parameters. To validate the effect, we developthe neural network with CHM layers that perform convolutional matching in thespace of translation and scaling. Our method sets a new state of the art onstandard benchmarks for semantic visual correspondence, proving its strongrobustness to challenging intra-class variations.