VALSE 论文速览 第175期：Distortion-aware Transformer in 360° SOD

论文题目：

Distortion-aware Transformer in 360° Salient Object Detection

作者列表：

赵寅捷 (北京航空航天大学)、赵立晨 (北京航空航天大学)、于茜 (北京航空航天大学)、张晶 (北京航空航天大学)、盛律 (北京航空航天大学)、徐东 (香港大学)

B站观看网址：

https://www.bilibili.com/video/BV1AM4m1R7BM/

论文摘要：

With the emergence of VR and AR, 360° data attracts increasing attention from the computer vision and multimedia communities. Typically, 360° data is projected into 2D ERP (equirectangular projection) images for feature extraction. However, existing methods cannot handle the distortions that result from the projection, hindering the development of 360-data-based tasks. Therefore, in this paper, we propose a Transformer-based model called DATFormer to address the distortion problem. We tackle this issue from two perspectives. Firstly, we introduce two distortion-adaptive modules. The first is a Distortion Mapping Module, which guides the model to pre-adapt to distorted features globally. The second module is a Distortion-Adaptive Attention Block that reduces local distortions on multi-scale features. Secondly, to exploit the unique characteristics of 360° data, we present a learnable relation matrix and use it as part of the positional embedding to further improve  performance. Extensive experiments are conducted on three public datasets, and the results show that our model outperforms existing 2D SOD (salient object detection) and 360 SOD methods.

参考文献：

[1] Zhao, Yinjie, et al. "Distortion-aware Transformer in 360° Salient Object Detection." Proceedings of the 31st ACM International Conference on Multimedia. 2023.

论文链接：

[https://arxiv.org/abs/2308.03359]

代码链接：

[https://github.com/yjzhao19981027/DATFormer]

视频讲者简介：

Yinjie Zhao received the BE degree from Beihang University, China in 2021. He is currently studying for a master's degree in Beihang University. His current research interests include 360-degree video and SOD tasks.

特别鸣谢本次论文速览主要组织者：

月度轮值AC：胡迪 (中国人民大学)