VALSE 论文速览 第184期：Personalization-based Backdoor

论文题目：

Personalization as  a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion  Models

作者列表：

黄怿豪 (南洋理工大学)，徐觉非 (纽约大学)，郭青 (Astar)，张杰 (南洋理工大学)，吴与桐 (南洋理工大学），胡铭 (南洋理工大学)，李恬霖 (南洋理工大学)，蒲戈光 (华东师范大学)，刘杨 (南洋理工大学)

B站观看网址：

https://www.bilibili.com/video/BV1sb421p77c/

论文摘要：

Although recent  personalization methods have democratized high-resolution image synthesis by  enabling swift concept acquisition with minimal examples and lightweight  computation, they also present an exploitable avenue for highly accessible  backdoor attacks. This paper investigates a critical and unexplored aspect of  text-to-image (T2I) diffusion models - their potential vulnerability to  backdoor attacks via personalization. By studying the prompt processing of  popular personalization methods (epitomized by Textual Inversion and  DreamBooth), we have devised dedicated personalization-based backdoor attacks  according to the different ways of dealing with unseen tokens and divide them  into two families: nouveau-token and legacy-token backdoor attacks. In comparison  to conventional backdoor attacks involving the fine-tuning of the entire  text-to-image diffusion model, our proposed personalization-based backdoor  attack method can facilitate more tailored, efficient, and few-shot attacks.  Through comprehensive empirical study, we endorse the utilization of the  nouveau-token backdoor attack due to its impressive effectiveness,  stealthiness, and integrity, markedly outperforming the legacy-token backdoor  attack.

论文链接：

[https://arxiv.org/abs/2305.10701]

代码链接：

[https://github.com/Huang-yihao/Personalization-based_backdoor]

视频讲者简介：

黄怿豪，南洋理工大学CSL实验室博后。主要研究方向是DeepFake，对抗攻击等。

特别鸣谢本次论文速览主要组织者：

月度轮值AC：周天飞 (北京理工大学)