Skip to content

Latest commit

 

History

History
66 lines (55 loc) · 5.66 KB

README.md

File metadata and controls

66 lines (55 loc) · 5.66 KB
Raw

Selective Frequency Network for Image Restoration (ICLR2023)

Yuning Cui, Yi Tao, Zhenshan Bing, Wenqi Ren, Xinwei Gao, Xiaochun Cao, Kai Huang, Alois Knoll

Update

Our extended paper has been accepted by T-PAMI (Image Restoration via Frequency Selection)

  • 2023.08.04 🎉 Release the model and code for image deraining
  • 2023.04.19 🎉 Release our results on RSBlur
  • 2023.04.14 :tada: Release pre-trained models, code for three tasks (dehazing, motion deblurring, desnowing), and resulting images on most datasets.

Image restoration aims to reconstruct the latent sharp image from its corrupted counterpart. Besides dealing with this long-standing task in the spatial domain, a few approaches seek solutions in the frequency domain in consideration of the large discrepancy between spectra of sharp/degraded image pairs. However, these works commonly utilize transformation tools, e.g., wavelet transform, to split features into several frequency parts, which is not flexible enough to select the most informative frequency component to recover. In this paper, we exploit a multi-branch and content-aware module to decompose features into separate frequency subbands dynamically and locally, and then accentuate the useful ones via channel-wise attention weights. In addition, to handle large-scale degradation blurs, we propose an extremely simple decoupling and modulation module to enlarge the receptive field via global and window-based average pooling. Integrating two developed modules into a U-Net backbone, the proposed Selective Frequency Network (SFNet) performs favorably against state-of-the-art algorithms on five image restoration tasks, including single-image defocus deblurring, image dehazing, image motion deblurring, image desnowing, and image deraining.

Architecture

Installation

The project is built with PyTorch 3.8, PyTorch 1.8.1. CUDA 10.2, cuDNN 7.6.5 For installing, follow these instructions:

conda install pytorch=1.8.1 torchvision=0.9.1 -c pytorch
pip install tensorboard einops scikit-image pytorch_msssim opencv-python

Install warmup scheduler:

cd pytorch-gradual-warmup-lr/
python setup.py install
cd ..

Training and Evaluation

Please refer to respective directories.

Results

Task Dataset PSNR SSIM Resulting Images
Motion Deblurring GoPro 33.27 0.963 gdrive,Baidu
HIDE 31.10 0.941 gdrive,Baidu
RSBlur 34.35 0.872 gdrive,Baidu
Image Dehazing SOTS-Indoor 41.24 0.996 gdrive,Baidu
SOTS-Outdoor 40.05 0.996 gdrive,Baidu
Dense-Haze 17.46 0.578 gdrive,Baidu
Image Desnowing CSD 38.41 0.99 gdrive,Baidu
SRRS 32.40 0.98 gdrive,Baidu
Snow100K 33.79 0.95 gdrive,Baidu
Image Deraining Average 33.56 0.929 gdrive,Baidu
Defocus Deblurring DPDDsingle 26.23 0.811 gdrive,Baidu
DPDDdual 26.34 0.817 gdrive,Baidu

Citation

If you find this project useful for your research, please consider citing:

@inproceedings{cui2023selective,
  title={Selective Frequency Network for Image Restoration},
  author={Cui, Yuning and Tao, Yi and Bing, Zhenshan and Ren, Wenqi and Gao, Xinwei and Cao, Xiaochun and Huang, Kai and Knoll, Alois},
  booktitle={The Eleventh International Conference on Learning Representations},
  year={2023}
}

Acknowledgement

This project is mainly based on MIMO-UNet.

Contact

Should you have any question, please contact Yuning Cui.