Image dehazing aims to recover a clean image from a hazy image, which is a challengingly longstanding problem. In this paper, we propose an Ensemble Multi-scale Residual Attention Network (EMRA-Net) to directly generate a clean image, which include two parts: a three-scale residual attention CNN (TRA-CNN), and an ensemble attention CNN (EA-CNN). In TRA-CNN, we employ wavelet transform to obtain the downsampled images, instead of using common spatial downsampling methods, such as nearest downsampling and strided-convolution. With the help of wavelet transform, we can avoid the loss of image texture details. Moreover, in each scale-branch, Res2Net modules are connected in series to make full use of the hierarchical features from the original hazy images, and channel attention mechanism is introduced to focus channel-dimension information. Finally, an EA-CNN is proposed to fuse coarse images generated from TRA-CNN into a refined clean image. Extensive experiments on the benchmark synthetic hazy datasets and the real-world hazy dataset prove that proposed EMRA-Net is superior to previous state-of-the-art methods both in subjective visual perception and objective image quality assessment metrics.