VALSE 首页 活动通知 查看内容

VALSE 论文速览 第89期:β-DARTS: 用于可微分架构搜索的Beta-Decay正则化 ...

2022-7-29 15:13| 发布者: 程一-计算所| 查看: 82| 评论: 0

摘要: 为了使得视觉与学习领域相关从业者快速及时地了解领域的最新发展动态和前沿技术进展,VALSE最新推出了《论文速览》栏目,将在每周发布一至两篇顶会顶刊论文的录制视频,对单个前沿工作进行细致讲解。本期VALSE论文速 ...


论文题目:β-DARTS: Beta-Decay Regularization for Differentiable Architecture Search

作者列表:Peng Ye (Fudan University)、Baopu Li (BAIDU USA LLC)、Yikang Li (Shanghai AI Laboratory)、Tao Chen (Fudan University)、Jiayuan Fan (Fudan University)、Wanli Ouyang (The University of Sydney, SenseTime Computer Vision Group, Australia)



Neural Architecture Search (NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural network automatically. Among them, differential NAS approaches such as DARTS, have gained popularity for the search efficiency. However, they suffer from two main issues, the weak robustness to the performance collapse and the poor generalization ability of the searched architectures. To solve these two problems, a simple-but-efficient regularization method, termed as Beta-Decay, is proposed to regularize the DARTS-based NAS searching process. Specifically, Beta-Decay regularization can impose constraints to keep the value and variance of activated architecture parameters from too large. Furthermore, we provide in-depth theoretical analysis on how it works and why it works. Experimental results on NAS-Bench-201 show that our proposed method can help to stabilize the searching process and makes the searched network more transferable across different datasets. In addition, our search scheme shows an outstanding property of being less dependent on training time and data. Comprehensive experiments on a variety of search spaces and datasets validate the effectiveness of the proposed method. The code is available at /Sunshine-Ye/Beta-DARTS .


[1] P. Ye, B. Li, Y. Li, T. Chen, J. Fan, W. Ouyang, “Beta-decay regularization for differentiable architecture search,”Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR oral), in press, 2022.






叶鹏,复旦大学EDL lab博士研究生,主要研究方向是计算机视觉、模型轻量化和神经架构搜索。


月度轮值AC:王智慧 (大连理工大学)、杨旭 (西安电子科技大学)

季度责任AC:魏秀参 (南京理工大学)





2、VALSE Webinar活动通常每周三晚上20:00进行,但偶尔会因为讲者时区问题略有调整,为方便您参加活动,请关注VALSE微信公众号:valse_wechat 或加入VALSE QQ R群,群号:137634472);

*注:申请加入VALSE QQ群时需验证姓名、单位和身份缺一不可。入群后,请实名,姓名身份单位。身份:学校及科研单位人员T;企业研发I;博士D;硕士M。


4您也可以通过访问VALSE主页: 直接查看Webinar活动信息。Webinar报告的PPT(经讲者允许后),会在VALSE官网每期报告通知的最下方更新。

小黑屋|手机版|Archiver|Vision And Learning SEminar

GMT+8, 2022-8-13 12:14 , Processed in 0.014579 second(s), 14 queries .

Powered by Discuz! X3.4

Copyright © 2001-2020, Tencent Cloud.