Self-Attention for Audio Super-Resolution

Convolutions operate only locally, thus failing to model global interactions.Self-attention is, however, able to learn representations that capturelong-range dependencies in sequences. We propose a network architecture foraudio super-resolution that combines convolution and self-attention.Attention-based Feature-Wise Linear Modulation (AFiLM) uses self-attentionmechanism instead of recurrent neural networks to modulate the activations ofthe convolutional model. Extensive experiments show that our model outperformsexisting approaches on standard benchmarks. Moreover, it allows for moreparallelization resulting in significantly faster training.