Image demoiréing is a complex image-restoration task because of the color and shape variations of moiré patterns. With the development of mobile devices, mobile phones can now be used to capture images at multiple resolutions. This difficulty increases when attempting to remove moiré from both low- and high-resolution images, as different resolutions make it challenging for existing methods to match the scales and textures of moiré. To solve these problems, we built a mixed attention residual module (MARM) by combining multi-scale feature extraction and mixed attention methods. Based on MARM, we propose a multi-scale adaptive mixed attention network (MA2Net) that can adapt to input images of different sizes and remove moiré of various shapes. Our model achieved the best results on four public datasets with resolutions ranging from 256×256 to 4k. Extensive experiments demonstrated the effectiveness of our model, which outperformed state-of-the-art methods by a large margin. We also conducted experiments on image deraining to validate the effectiveness of our model in other image-restoration tasks, and MA2Net achieved state-of-the-art performance on the Rain200H dataset.