VALSE 论文速览 第132期：IRRA：基于隐式关系推理的文本图像跨模态行人重识别 ...

论文题目：Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval

作者列表：

姜定 (武汉大学)、叶茫 (武汉大学)

B站观看网址：

https://www.bilibili.com/video/BV1tp4y1M7yn/

论文摘要：

Text-to-image person retrieval aims to identify the target person based on a given textual description query. The primary challenge is to learn the mapping of visual and textual modalities into a common latent space. Prior works have attempted to address this challenge by leveraging separately pre-trained unimodal models to extract visual and textual features. However, these approaches lack the necessary underlying alignment capabilities required to match multimodal data effectively. Besides, these works use prior information to explore explicit part alignments, which may lead to the distortion of intra-modality information. To alleviate these issues, we present IRRA: a cross-modal Implicit Relation Reasoning and Aligning framework that learns relations between local visual-textual tokens and enhances global image-text matching without requiring additional prior supervision. Specifically, we first design an Implicit Relation Reasoning module in a masked language modeling paradigm. This achieves cross-modal interaction by integrating the visual cues into the textual tokens with a cross-modal multimodal interaction encoder. Secondly, to globally align the visual and textual embeddings, Similarity Distribution Matching is proposed to minimize the KL divergence between image-text similarity distributions and the normalized label matching distributions. The proposed method achieves new state-of-the-art results on all three public datasets, with a notable margin of about 3%-9% for Rank-1 accuracy compared to prior methods.

论文信息：

[1] Ding Jiang and Mang Ye. 2023. Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

论文链接：

[https://arxiv.org/abs/2303.12501]

代码链接：

[https://github.com/anosorae/IRRA]

视频讲者简介：

姜定，武汉大学计算机学院硕士生。主要研究方向为跨模态检索。

个人主页：

https://marswhu.github.io/

特别鸣谢本次论文速览主要组织者：

月度轮值AC：叶茫 (武汉大学)