VALSE 论文速览 第91期：DF-GAN：一个简单有效的文本到图像生成对抗网络 ...

论文题目：DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis

作者列表：Ming Tao (Nanjing University of Posts and Telecommunications), Hao Tang (ETH Zurich), Fei Wu (Nanjing University of Posts and Telecommunications), Xiaoyuan Jing (Wuhan University), Bing-Kun Bao (Nanjing University of Posts and Telecommunications), Changsheng Xu (NLPR, Institute of Automation, CAS)

B站观看网址：

https://www.bilibili.com/video/BV1ad4y1D76D/

论文摘要：

Synthesizing high-quality realistic images from text descriptions is a challenging task. Existing text-to-image Generative Adversarial Networks generally employ a stacked architecture as the backbone yet still remain three flaws. First, the stacked architecture introduces the entanglements between generators of different image scales. Second, existing studies prefer to apply and fix extra networks in adversarial learning for text-image semantic consistency, which limits the supervision capability of these networks. Third, the cross-modal attention-based text-image fusion that widely adopted by previous works is limited on several special image scales because of the computational cost. To these ends, we propose a simpler but more effective Deep Fusion Generative Adversarial Networks (DF-GAN).

To be specific, we propose:

(i) a novel one-stage text-to-image backbone that directly synthesizes high-resolution images without entanglements between different generators,

(ii) a novel Target-Aware Discriminator composed of Matching-Aware Gradient Penalty and One-Way Output, which enhances the text-image semantic consistency without introducing extra networks,

(iii) a novel deep text-image fusion block, which deepens the fusion process to make a full fusion between text and visual features.

Compared with current state-of-the-art methods, our proposed DF-GAN is simpler but more efficient to synthesize realistic and text-matching images and achieves better performance on widely used datasets.

论文信息：

[1] Ming Tao, Hao Tang, Fei Wu, Xiaoyuan Jing, Bing-Kun Bao and Changsheng Xu. DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis. CVPR 2022 (Oral)

论文链接：

[https://arxiv.org/abs/2008.05865]

代码链接：

[https://github.com/tobran/DF-GAN]

视频讲者简介：

陶明，南京邮电大学博士生，主要研究方向是深度学习与多模态生成。

特别鸣谢本次论文速览主要组织者：

月度轮值AC：王智慧 (大连理工大学)、杨旭 (西安电子科技大学)

季度责任AC：魏秀参 (南京理工大学)