VALSE 论文速览 第139期：三维场景中基于扩散模型的生成、优化、与规划 ...

论文题目：Diffusion-based Generation, Optimization, and Planning in 3D Scenes

作者列表：

黄思远* (北京通用人工智能研究院)，王赞* (北京理工大学)，李浦豪 (清华大学)、贾宝雄 (北京通用人工智能研究院)、刘腾宇 (北京通用人工智能研究院)、朱毅鑫 (北京大学)、梁玮 (北京理工大学)、朱松纯 (北京通用人工智能研究院)

B站观看网址：

https://www.bilibili.com/video/BV1Rw411r7uW/

论文摘要：

We introduce SceneDiffuser, a conditional generative model for 3D scene understanding. SceneDiffuser provides a unified model for solving scene-conditioned generation, optimization, and planning. In contrast to prior works, SceneDiffuser is intrinsically scene-aware, physics-based, and goal-oriented. With an iterative sampling strategy, SceneDiffuser jointly formulates the scene-aware generation, physics-based optimization, and goal-oriented planning via a diffusion-based denoising process in a fully differentiable fashion. Such a design alleviates the discrepancies among different modules and the posterior collapse of previous scene-conditioned generative models. We evaluate SceneDiffuser with various 3D scene understanding tasks, including human pose and motion generation, dexterous grasp generation, path planning for 3D navigation, and motion planning for robot arms. The results show significant improvements compared with previous models, demonstrating the tremendous potential of SceneDiffuser for the broad community of 3D scene understanding.

论文信息：

[1] Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, Song-Chun Zhu, Diffusion-based Generation, Optimization, and Planning in 3D Scenes, In CVPR, 2023.

论文链接：

[https://arxiv.org/abs/2301.06015]

代码链接：

[https://github.com/scenediffuser/Scene-Diffuser]

视频讲者简介：

Zan Wang is a second-year Ph.D. candidate in the School of Computer Science, Beijing Institute of Technology (BIT), Beijing. He is currently working in Visual Perception & Human-computer Interaction Lab (PI Lab), advised by Prof. Wei Liang. He has been interning at Beijing Institute for General Artificial Intelligence (BIGAI) from Oct. 2021 to now, under the supervision of vision team leader, Dr. Siyuan Huang. Before beginning his Ph.D. study, he received B.S. in Internet of Things Engineering from Beijing Institute of Technology (BIT) in June 2020. His research interests lie in the field of Computer Vision, Graphics, and Deep Learning.

个人主页：

https://silvester.wang/

特别鸣谢本次论文速览主要组织者：

月度轮值AC：焦剑波 (伯明翰大学)

季度轮值AC：叶茫 (武汉大学)