VALSE Webinar 2024-24期 总第359期 医学多模态分析与研究：从传统模型到大模型的演变 ...

报告嘉宾：刘明霞  (北卡罗来纳大学教堂山分校)

报告题目：多中心多模态脑影像智能分析及应用研究

报告嘉宾：王淑君 (香港理工大学)

报告题目：Multi-modal Representation Learning for Medical Data Analysis

Panel议题：

1. 在大模型时代，相较于传统模型，医学多模态分析面临哪些新挑战？

2. 鉴于大模型依赖海量数据，而医学多模态数据获取困难，有哪些方法和技术可以有效解决？

3. 医学多模态大模型分析涉及大量3D数据 (e.g., 3D MRI, CT, PET等)，而学术界的计算资源有限，您最看好哪些技术或方法来解决这个问题？

4. 大模型容易出现幻象 (Hallucination)，这对医学多模态大模型的落地应用带来极大的挑战，有哪些方法可以有效解决？

5. 大模型时代医学多模态分析的发展趋势是什么？有哪些潜在研究方向？

Panel嘉宾：

刘明霞  (北卡罗来纳大学教堂山分校)、王淑君 (香港理工大学)、周泸萍 (悉尼大学)、Paul Pu Liang (麻省理工大学)

*欢迎大家在下方留言提出主题相关问题，主持人和panel嘉宾会从中选择若干热度高的问题加入panel议题！

报告嘉宾：刘明霞 (北卡罗来纳大学教堂山分校)

报告时间：2024年8月28日 (星期三)晚上20:00 (北京时间)

报告题目：多中心多模态脑影像智能分析及应用研究

报告人简介：

刘明霞，助理教授，美国北卡罗来纳大学教堂山分校。2015年在南京航空航天大学计算机科学与技术学院获得博士学位。2014年至2017在美国北卡罗来纳大学教堂山分校先后以研究助理和博士后研究员身份进行研究工作。目前主要从事机器学习、模式识别和医学图像分析等领域的研究工作。目前担任权威期刊Pattern Recognition、Medical Image Analysis、Neural Networks和 IEEE Transactions on Cognitive and Developmental Systems等著名杂志的副主编；在IEEE Transactions on Pattern Analysis and Machine Intelligence、Nature Communications、IEEE Transactions on Cybernetics、IEEE Transaction on Medical Imaging、Medical Image Analysis和AAAI等国顶级期刊和会议上发表论文200余篇；相关研究工作被谷歌学术引用8000余次。曾获国际医学图像计算与计算机辅助介入学会（MICCAI）颁发的青年学者成就提名奖、中国人工智能学会颁发的优秀博士学位论文提名奖、以及江苏省计算机学会颁发的优秀博士学位论文奖等。

个人主页：

https://mingxia.web.unc.edu/

报告摘要：

Multi-site multi-modal neuroimaging data, such as magnetic resonance imaging (MRI) and positron emission tomography (PET), are critical to expanding the diversity of subject populations and enhancing the statistical robustness of predictive models in neuroscience research. Despite their potential, the field faces substantial challenges, notably the heterogeneity of data across imaging sites and modalities. Addressing these complexities, my research focuses on creating machine learning and deep learning methodologies to analyze multi-modal imaging data from multiple sites, with the goal of uncovering imaging biomarkers associated with neurodegenerative disorders. This talk will delineate our progress in address three long-standing challenges: neuroimage representation learning, multimodality neuroimage fusion, and multi-site data adaptation. Key highlights will include our latest advances in the representation learning of MRI, capturing both structural and functional dimensions. Subsequently, I will elucidate our strategies for the effective integration of multi-modal neuroimaging data, which promises the accurate synthesis of MRI and PET scans, particularly beneficial in cases plagued by missing or incomplete data modalities. Concluding the talk, I will introduce our comprehensive suite of multi-site neuroimage harmonization techniques and unveil DomainATM, our open-source toolbox specifically designed for medical data adaptation.

报告嘉宾：王淑君 (香港理工大学)

报告时间：2024年8月28日 (星期三)晚上20:30 (北京时间)

报告题目：Multi-modal Representation Learning for Medical Data Analysis

报告人简介：

Dr Emma Shujun WANG is an Assistant Professor at PolyU BME.  Before that, she was a Research Associate in the Department of Applied Mathematics and Theoretical Physics, at the University of Cambridge from 2022 to 2023. She was a Postdoctoral Researcher in the Department of Computer Science and Engineering at The Chinese University of Hong Kong from 2021 to 2022. She received her Ph.D from the Department of Computer Science and Engineering at The Chinese University of Hong Kong in 2021, and B.Eng from Honors College at Northwestern Polytechnical University in 2017. Dr Wang has published around 30 papers on top-tier conferences and journals (on The Lancet Digital Health (IF: 36.615), IEEE-TMI, MedIA, NeurIPS, AAAI, ECCV, MICCAI, etc.).

个人主页：

https://www.polyu.edu.hk/en/bme/people/academic-staff/dr-emma-wang/

报告摘要：

We delve into the intricate challenges of analyzing medical data and explore the core difficulties inherent in multi-modal learning. The talk will highlight two significant applications of multi-modal representation learning: one focused on chest X-ray interpretation and another on Alzheimer’s disease classification. We will discuss how leveraging diverse data types can enhance diagnostic accuracy and predictive modeling in these areas, addressing both the potential and obstacles of integrating multi-modality medical datasets. This comprehensive analysis aims to advance our understanding of complex diseases and improve patient outcomes through innovative computational approaches.

参考文献：

[1] Li, Qingqiu, et al. "Anatomical Structure-Guided Medical Vision-Language Pre-training." MICCAI (2024).

[2] Yang, Guangqian, et al. "CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification." arXiv:2403.16520 (2024).

Panel 嘉宾：周泸萍 (悉尼大学)

嘉宾简介：

A/Prof Luping Zhou is the Director of Software Engineering in School of Electrical and Computer Engineering (ECE), the University of Sydney.  She obtained her PhD from Australian National University, and was a recipient of ARC (Australian Research Council) DECRA award (Discovery Early Career Researcher Award). A/Prof Zhou is co-leading the Digital Health Imaging (DHI) research theme in Faculty of Engineering, USyd, which is devoted to promoting the synergy of USyd’s strengths and leadership in AI and medical imaging and building trust with the industry for impactful research. She is also associated with Sydney Artificial Intelligence Centre. A/Prof Zhou has a broad research interest in medical image analysis, machine learning, and computer vision. She has published 150+ scientific papers in these fields, including those in the top-ranked outlets such as IEEE TPAMI, IEEE TMI, Medical Image Analysis (MedIA), MICCAI, CVPR, ICCV, ECCV. A/Prof Zhou is the Associate Editor of IEEE TMI and MedIA. She served MICCAI as an Area Chair (2020-2022, 2024), Oral Session Chair (2021), and Young Scientist Award selection committee (2022). She is also in the organizing committee of MICCAI 2019 and MICCAI 2025. A/Prof Zhou is an Area Chair of ECCV 2024. She is a Senior member of IEEE.

个人主页：

https://sites.google.com/view/lupingzhou

Panel 嘉宾：Paul Pu Liang (麻省理工大学)

嘉宾简介：

Paul Liang is an Assistant Professor at Massachusetts Institute of Technology Media Lab and EECS. His research advances the foundations of multisensory artificial intelligence to enhance the human experience. He is a recipient of the Siebel Scholars Award, Waibel Presidential Fellowship, Facebook PhD Fellowship, Center for ML and Health Fellowship, Rising Stars in Data Science, and 3 best paper awards. Outside of research, he received the Alan J. Perlis Graduate Student Teaching Award for instructing courses on multimodal machine learning.

个人主页：

https://pliang279.github.io/

主持人：屈靓琼 (香港大学)

主持人简介：

Dr. Liangqiong Qu is an Assistant Professor at the Department of Statistics and Actuarial Science, the University of Hong Kong. Before joining HKU, she was a postdoctoral fellow at Stanford University and the University of North Carolina at Chapel Hill. Her research interests span the area of artificial intelligence, computer vision and medical imaging processing. Her research results have been published in 2 book chapters, and 40 peer-reviewed articles including top-tier venues such as CVPR, PNAS, MedIA, TMI, EMNLP, Nature Method, Cancer Cell, Nature Machine Intelligence, MICCAI. She has received the Best Paper Award at the Conference on Health, Inference, and Learning (CHIL) 2023, and was also awarded the Natural Science Academic Achievement Award of Liaoning Province (1st).  

个人主页：

https://liangqiong.github.io/

特别鸣谢本次Webinar主要组织者：

主办AC：屈靓琼 (香港大学)