Video deblurring has achieved remarkable progress thanks to the success ofdeep neural networks. Most methods solve for the deblurring end-to-end withlimited information propagation from the video sequence. However, differentframe regions exhibit different characteristics and should be provided withcorresponding relevant information. To achieve fine-grained deblurring, wedesigned a memory branch to memorize the blurry-sharp feature pairs in thememory bank, thus providing useful information for the blurry query input. Toenrich the memory of our memory bank, we further designed a bidirectionalrecurrency and multi-scale strategy based on the memory bank. Experimentalresults demonstrate that our model outperforms other state-of-the-art methodswhile keeping the model complexity and inference time low. The code isavailable at https://github.com/jibo27/MemDeblur.