VALSE 论文速览 第195期：F3-Pruning

论文题目：

F3-Pruning: A Training-Free and  Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis

作者列表：

苏思桐 (电子科技大学)，刘健知 (电子科技大学)，高联丽 (电子科技大学)，宋井宽 (电子科技大学)

B站观看网址：

https://www.bilibili.com/video/BV11GxPerEDf/?spm_id_from=333.999.0.0&vd_source=60f8dec9131b788976f17e2922918e04

复制链接到浏览器打开或点击阅读原文即可跳转至观看页面。

论文摘要：

Recently Text-to-Video (T2V) synthesis has undergone a breakthrough by training transformers or diffusion models on large-scale datasets. Nevertheless, inferring such large models incurs huge costs. Previous inference acceleration works either require costly retraining or are model-specific. To address this issue, instead of retraining we explore the inference process of two mainstream T2V models using transformers and diffusion models. The exploration reveals the redundancy in temporal attention modules of both models, which are commonly utilized to establish temporal relations among frames. Consequently, we propose a training-free and generalized pruning strategy called F3-Pruning to prune redundant temporal attention weights. Specifically, when aggregate temporal attention values are ranked below a certain ratio, corresponding weights will be pruned. Extensive experiments on three datasets using a classic transformer-based model CogVideo and a typical diffusion-based model Tune-A-Video verify the effectiveness of F3-Pruning in inference acceleration, quality assurance and broad applicability.

参考文献：

[1] Sitong Su*, Jianzhi Liu*, Lianli Gao, Jingkuan Song. F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesis. The 38th Annual AAAI Conference on Artificial Intelligence (AAAI), Vancouver, Canada, 2024.

论文链接：

[https://arxiv.org/abs/2312.03459]

视频讲者简介：

Sitong Su is working toward the Ph.D. degree in the School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China. Her research interests manly focus on computer vision, cross-modal visual synthesis and edit, diffusion models.

特别鸣谢本次论文速览主要组织者：

月度轮值AC：王一帆 (大连理工大学)