VALSE 论文速览 第179期：Neural Interactive Keypoint Detection

论文题目：

Neural Interactive Keypoint Detection

作者列表：

Jie Yang (香港中文大学 (深圳)), Ailing Zeng* (IDEA研究院), Feng Li (IDEA研究院), Shilong Liu (IDEA研究院), Ruimao Zhang* (香港中文大学 (深圳)), Lei Zhang (IDEA研究院)

B站观看网址：

https://www.bilibili.com/video/BV1PM4m167nD/

论文摘要：

This work proposes an end-to-end neural interactive keypoint detection framework named Click-Pose, which can significantly reduce more than 10 times labeling costs of 2D keypoint annotation compared with manual-only annotation. Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process. Specifically, we design the pose error modeling strategy that inputs the ground truth pose combined with four typical pose errors into the decoder and trains the model to reconstruct the correct poses, which enhances the self-correction ability of the model. Then, we attach an interactive human-feedback loop that allows receiving users’ clicks to correct one or several predicted keypoints and iteratively utilizes the decoder to update all other keypoints with a minimum number of clicks (NoC) for efficient annotation. We validate Click-Pose in in-domain, out-of-domain scenes, and a new task of keypoint adaptation. For annotation, Click-Pose only needs 1.97 and 6.45 NoC@95 (at precision 95%) on COCO and Human-Art, reducing 31.4% and 36.3% efforts than the SOTA model with manual correction, respectively. Besides, without user clicks, Click-Pose surpasses the previous end-to-end model by 1.4 AP on COCO and 3.0 AP on Human-Art.

论文链接：

[https://openaccess.thecvf.com/content/ICCV2023/papers/Yang_Neural_Interactive_Keypoint_Detection_ICCV_2023_paper.pdf]

代码链接：

[https://github.com/IDEA-Research/Click-Pose]

视频讲者简介：

杨杰，香港中文大学深圳，二年级博士生；导师为张瑞茂教授，研究方向为以人为中心的理解，感知和生成，在CVPR,ICCV,ICLR,NeurIPS等会议上发表多篇文章。

特别鸣谢本次论文速览主要组织者：

月度轮值AC：张瑞茂 (香港中文大学 (深圳))