VALSE 论文速览 第204期：病理图驱动的的细粒度脑CT报告生成研究 ...

论文题目：

Granularity Matters: Pathological Graph-driven Cross-modal Alignment for Brain CT Report Generation

作者列表：

时彦钊 (北京工业大学)，冀俊忠 (北京工业大学)，张晓丹 (北京工业大学)，屈靓琼 (香港大学)，刘颖 (北京大学第三医院)

B站观看网址：

https://www.bilibili.com/video/BV1oXxNeAE9t/?spm_id_from=333.999.0.0&vd_source=2044ac986b5caa4c8b5f00b525441df3

复制链接到浏览器打开或点击阅读原文即可跳转至观看页面。

论文摘要：

The automatic Brain CT reports generation can improve the efficiency and accuracy of diagnosing cranial diseases. However, current methods are limited by 1) coarse-grained supervision: the training data in image-text format lacks detailed supervision for recognizing subtle abnormalities, and 2) coupled cross-modal alignment: visual-textual alignment may be inevitably coupled in a coarse-grained manner, resulting in tangled feature representation for report generation. In this paper, we propose a novel Pathological Graph-driven Cross-modal Alignment (PGCA) model for accurate and robust Brain CT report generation. Our approach effectively decouples the cross-modal alignment by constructing a Pathological Graph to learn fine-grained visual cues and align them with textual words. This graph comprises heterogeneous nodes representing essential pathological attributes (i.e., tissue and lesion) connected by intra- and inter-attribute edges with prior domain knowledge. Through carefully designed graph embedding and updating modules, our model refines the visual features of subtle tissues and lesions and aligns them with textual words using contrastive learning. Extensive experimental results confirm the viability of our method. We believe that our PGCA model holds the potential to greatly enhance the automatic generation of Brain CT reports and ultimately contribute to improved cranial disease diagnosis.

参考文献：

[1] Yanzhao Shi, Junzhong Ji, Xiaodan Zhang, Liangqiong Qu, and Ying Liu. 2023. Granularity Matters: Pathological Graph-driven Cross-modal Alignment for Brain CT Report Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), pages 6617–6630.

论文链接：

[https://aclanthology.org/2023.emnlp-main.408.pdf]

视频讲者简介：

时彦钊，北京工业大学硕士研究生二年级在读，师从冀俊忠教授与张晓丹副教授，主要研究方向为医学报告自动生成、医学视觉文本表示学习。目前在国际顶级会议和SCI期刊上发表多篇论文。

个人主页：

https://yanzhaoshi.github.io/

特别鸣谢本次论文速览主要组织者：

月度轮值AC：武宇 (武汉大学)