VALSE 论文速览 第45期：Neural Body：用带有隐式结构编码的INR生成动态人体新视角 ...

论文题目：Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

作者列表：彭思达 (浙江大学)，张远青 (浙江大学)，徐英豪 (香港中文大学)，王倩倩 (康奈尔大学)，帅青 (浙江大学)，鲍虎军 (浙江大学)，周晓巍 (浙江大学)

B站观看网址：

https://www.bilibili.com/video/BV15F411p7H7/

论文摘要：

This paper addresses the challenge of novel view synthesis for a human performer from a very sparse set of camera views. Some recent works have shown that learning implicit neural representations of 3D scenes achieves remarkable view synthesis quality given dense input views. However, the representation learning will be ill-posed if the views are highly sparse. To solve this ill-posed problem, our key idea is to integrate observations over video frames. To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated. The deformable mesh also provides geometric guidance for the network to learn 3D representations more efficiently. To evaluate our approach, we create a multi-view dataset named ZJU-MoCap that captures performers with complex motions. Experiments on ZJU-MoCap show that our approach outperforms prior works by a large margin in terms of novel view synthesis quality. We also demonstrate the capability of our approach to reconstruct a moving person from a monocular video on the People-Snapshot dataset.

论文信息：

[1] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou, Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. In CVPR, 2021.

论文链接：

[https://openaccess.thecvf.com/content/CVPR2021/papers/Peng_Neural_Body_Implicit_Neural_Representations_With_Structured_Latent_Codes_for_CVPR_2021_paper.pdf]

代码链接：

[https://github.com/zju3dv/neuralbody]

视频讲者简介：

彭思达是浙江大学CAD&CG国家重点实验室四年级博士研究生，导师为周晓巍研究员。研究方向为三维视觉，主要研究三维重建与视角合成。博士至今以一作身份在TPAMI、CVPR、ICCV等会议或期刊发表5篇论文，论文引用超过500次。在2020年获得CCF-CV学术新锐奖，全国仅评选3人。在2021年一作论文入围CVPR Best Paper Candidates。发表论文均已开源，在GitHub上Star数超过2000次。

特别鸣谢本次论文速览主要组织者：

月度轮值AC：董宣毅 (Amazon)、谢凌曦 (华为数字技术有限公司)

季度责任AC：许永超 (武汉大学)