VALSE 论文速览 第172期：Learning to Upsample by Learning to Sample

论文题目：

Learning to Upsample by Learning to Sample

作者列表：

刘文泽 (华中科技大学)，陆昊 (华中科技大学)，付洪涛 (华中科技大学)，曹治国 (华中科技大学)

B站观看网址：

https://www.bilibili.com/video/BV1Sj42197KT/

论文摘要：

We present DySample, an ultra-lightweight and effective dynamic upsampler. While impressive performance gains have been witnessed from recent kernel-based dynamic upsamplers such as CARAFE, FADE, and SAPA, they introduce much workload, mostly due to the time-consuming dynamic convolution and the additional sub-network used to generate dynamic kernels. Further, the need for high-res feature guidance of FADE and SAPA somehow limits their application scenarios. To address these concerns, we bypass dynamic convolution and formulate upsampling from the perspective of point sampling, which is more resource-efficient and can be easily implemented with the standard built-in function in PyTorch. We first showcase a naive design, and then demonstrate how to strengthen its upsampling behavior step by step towards our new upsampler, DySample. Compared with former kernel-based dynamic upsamplers, DySample requires no customized CUDA package and has much fewer parameters, FLOPs, GPU memory, and latency. Besides the light-weight characteristics, DySample outperforms other upsamplers across five dense prediction tasks, including semantic segmentation, object detection, instance segmentation, panoptic segmentation, and monocular depth estimation.

参考文献：

[1] Wenze Liu, Hao Lu, Hongtao Fu, Zhiguo Cao. Learning to Upsample by Learning to Sample. IEEE International Conference on Computer Vision (ICCV), 2023.

论文链接：

[https://arxiv.org/abs/2308.15085]

代码链接：

[https://github.com/tiny-smart/dysample]

视频讲者简介：

Wenze Liu is currently a master student at Huazhong University of Science and Technology. He received the bachelor degree in Automation from Huazhong University of Science and Technology in 2021. His research interests include computer vision and pattern recognition. He published several articles in ECCV, NeurIPS and ICCV. He currently studies dense prediction and multimodal AI.

个人主页：

https://github.com/teleppo

特别鸣谢本次论文速览主要组织者：

月度轮值AC：李爽 (北京理工大学)