20201223-30 深度模型架构设计与优化

报告嘉宾：代季峰 (商汤)

报告题目：Deformable DETR: Deformable Transformers for End-to-End Object Detection

报告嘉宾：董宣毅 (University of Technology Sydney)

报告题目：Extending the Search from Architecture to Hyperparameter, Hardware, and System

Panel嘉宾：

谢凌曦 (华为)、代季峰 (商汤)、董宣毅 (University of Technology Sydney)、刘晨曦 (Waymo)

Panel议题：

1. 我们首先讨论手工设计的网络结构。回顾过去的一年，大家认为业界有哪些工作是值得仔细品味、学习的？它们可能会对深度学习带来怎样的启发？

2. 2020年，一个值得注意的现象是基于transformer的设计正在“入侵”计算机视觉领域。大家对与transformer在视觉领域的未来有什么看法？它是attention模型的一个简单推广，还是蕴含着可能颠覆传统的力量？

3. 我们再来讨论自动设计的网络结构。可以说，2020年AutoML领域的发展远未达到预期，不论是NAS还是HPO都遇到了很大的瓶颈。这是否意味着，先前业界对于AutoML的期待有所偏差？我们应该怎样看待AutoML这项技术？

4. 我们聚焦一个更具体的问题。不少研究者都提出，受限的搜索空间极大地约束了网络架构搜索的发展。大家对这个问题有什么看法？在怎样的方向上拓展搜索空间，有可能打破现有的僵局？

5. 最后让我们讨论一个炙手可热的话题：自监督学习。大家认为迅猛发展的自监督学习算法，会对手工或者自动设计的网络结构产生什么影响吗？反过来，AutoML技术是否会对自监督学习产生帮助？

*欢迎大家在下方留言提出主题相关问题，主持人和panel嘉宾会从中选择若干热度高的问题加入panel议题！

报告嘉宾：代季峰 (商汤)

报告时间：2020年12月23日(星期三)晚上20:00(北京时间)

报告题目：Deformable DETR: Deformable Transformers for End-to-End Object Detection

报告人简介：

代季峰，商汤研究院研究执行总监，北京市智源青年科学家。2009年和2014年分别获得清华大学自动化系本科和博士学位，2012年至2013年在加州大学洛杉矶分校VCLA实验室访学。2014年至2019年在微软亚洲研究院视觉计算组工作，曾担任首席研究主管（Principle Research Manager）。2019年至今在商汤研究院工作，担任研究执行总监。他在领域顶级会议和期刊上发表三十余篇论文，谷歌学术引用共计9000余次，曾经连续两年在计算机视觉领域权威的COCO物体识别竞赛中获得第一名。

个人主页：

https://jifengdai.org/

报告摘要：

DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow convergence and limited feature spatial resolution, due to the limitation of Transformer attention modules in processing image feature maps. To mitigate these issues, we proposed Deformable DETR, whose attention modules only attend to a small set of key sampling points around a reference. Deformable DETR can achieve better performance than DETR (especially on small objects) with 10× less training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach.

报告嘉宾：董宣毅 (University of Technology Sydney)

报告时间：2020年12月23日(星期三)晚上20:30(北京时间)

报告题目：Extending the Search from Architecture to Hyperparameter, Hardware, and System

报告人简介：

Xuanyi Dong is a Ph.D. student at the University of Technology Sydney. He has taken six internships in industrial companies, including Google Brain AutoML team and Facebook Reality Lab. During his Ph.D. study, he has published over 20 papers on CVPR/ICCV/TPAMI/etc with citations 1600+. He was awarded the 2019 Google Ph.D. Fellowship. He was one of the winners at TRECVID Video Localization 2016 and ranked second at ILSVRC Object Localization 2015.

个人主页：

https://xuanyidong.com/

报告摘要：

In the past years, neural architecture search has changed the game of architecture design and brought significant accuracy gain in many applications. Despite these remarkable breakthroughs, the headroom of architecture search is quickly narrowed down. In this talk, I will present our attempt for the future of the search, including the joint hyperparameter and architecture search, the hardware and architecture search, and the software system.

Panel嘉宾：刘晨曦 (Waymo)

嘉宾简介：

Chenxi Liu is a research scientist at Waymo. Before that, he received his Ph.D. in Computer Science from Johns Hopkins University, M.S. in Statistics from University of California, Los Angeles, and B.E. in Automation from Tsinghua University. He has also spent time at Facebook, Google, Adobe, Toyota Technological Institute at Chicago, University of Toronto, and Rice University. His main research focus is automated machine learning for computer vision.

个人主页：

https://www.cs.jhu.edu/~cxliu/

主持人：黄高 (清华大学)

主持人简介：

Gao Huang is currently an Assistant Professor with the Department of Automation, Tsinghua University. He received the B.S. degree in automation from Beihang University in 2009, and the Ph.D. degree in automation from Tsinghua University in 2015. He was a Post-Doctoral Researcher with the Department of Computer Science, Cornell University, from 2015 to 2018. His research interests include deep learning and computer vision.

个人主页：

http://www.gaohuang.net/

主持人：谢凌曦 (华为)

主持人简介：

Lingxi Xie is currently a senior researcher at Huawei Inc. He obtained B.E. and Ph.D. in engineering, both from Tsinghua University, in 2010 and 2015, respectively. He also served as a post-doctoral researcher at the CCVL lab from 2015 to 2019, having moved from the University of California, Los Angeles to the Johns Hopkins University. His research interests lie in computer vision, in particular, the application of automated deep learning algorithms.

个人主页：

http://lingxixie.com/

20-30期VALSE在线学术报告参与方式：

长按或扫描下方二维码，关注“VALSE”微信公众号 (valse_wechat)，后台回复“30期”，获取直播地址。

特别鸣谢本次Webinar主要组织者：

主办AC：黄高 (清华大学)

协办AC：谢凌曦 (华为)

责任SAC：欧阳万里 (悉尼大学)