VALSE 论文速览 第168期：Sparse Transformer for Image Deraining

论文题目：

Learning A Sparse Transformer Network for Effective Image Deraining

作者列表：

陈翔 (南京理工大学)，李灏 (南京理工大学)，李明强 (中电科信息科学研究院)，潘金山 (南京理工大学)

B站观看网址：

https://www.bilibili.com/video/BV17D42157Lc/

论文摘要：

Transformers-based methods have achieved significant performance in image deraining as they can model the non-local information which is vital for high-quality image reconstruction. In this paper, we find that most existing Transformers usually use all similarities of the tokens from the query-key pairs for the feature aggregation. However, if the tokens from the query are different from those of the key, the self-attention values estimated from these tokens also involve in feature aggregation, which accordingly interferes with the clear image restoration. To overcome this problem, we propose an effective DeRaining network, Sparse Transformer (DRSformer) that can adaptively keep the most useful self-attention values for feature aggregation so that the aggregated features better facilitate high-quality image reconstruction. Specifically, we develop a learnable top-k selection operator to adaptively retain the most crucial attention scores from the keys for each query for better feature aggregation. Simultaneously, as the naive feed-forward network in Transformers does not model the multi-scale information that is important for latent clear image restoration, we develop an effective mixed-scale feed-forward network to generate better features for image deraining. To learn an enriched set of hybrid features, which combines local context from CNN operators, we equip our model with mixture of experts feature compensator to present a cooperation refinement deraining scheme. Extensive experimental results on the commonly used benchmarks demonstrate that the proposed method achieves favorable performance against state-of-the-art approaches.

参考文献：

[1] Xiang Chen, Hao Li, Mingqiang Li, Jinshan Pan, “Learning A Sparse Transformer Network for Effective Image Deraining”, in Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2023), Vancouver, Canada, June 2023.

论文链接：

[https://openaccess.thecvf.com/content/CVPR2023/papers/Chen_Learning_a_Sparse_Transformer_Network_for_Effective_Image_Deraining_CVPR_2023_paper.pdf]

代码链接：

[https://github.com/cschenxiang/DRSformer]

视频讲者简介：

陈翔，南京理工大学博士生，师从潘金山教授，主要研究方向为图像去雨。以第一作者在CVPR、ECCV、AAAI等国际高水平会议上发表多篇论文，曾获华为终端Camera学术之星等多项荣誉。

个人主页：

https://cschenxiang.github.io/

特别鸣谢本次论文速览主要组织者：

月度轮值AC：胡迪 (中国人民大学)