VALSE 论文速览 第131期：EA-HAS-Bench:能量感知超参数与网络架构搜索基准 ...

论文题目：EA-HAS-Bench: Energy-aware Hyperparameter and Architecture Search Benchmark

作者列表：

窦曙光 (同济大学)，蒋忻洋 (微软亚洲研究院)，赵才荣 (同济大学)，李东胜 (微软亚洲研究院)

B站观看网址：

https://www.bilibili.com/video/BV1NN4y1Z7x7/

论文摘要：

The energy consumption for training deep learning models is increasing at an alarming rate due to the growth of training data and model scale, resulting in a negative impact on carbon neutrality. Energy consumption is an especially pressing issue for AutoML algorithms because it usually requires repeatedly traininglarge numbers of computationally intensive deep models to search for optimal configurations. This paper takes one of the most essential steps in developing energy-aware (EA) NAS methods, by providing a benchmark that makes EANAS research more reproducible and accessible. Specifically, we present the first large-scale energy-aware benchmark that allows studying AutoML methods to achieve better trade-offs between performance and search energy consumption, named EA-HAS-Bench. EA-HAS-Bench provides a large-scale architecture/hyperparameter joint search space, covering diversified configurations related to energy consumption. Furthermore, we propose a novel surrogate model specially designed for large joint search space, which proposes a Bezier curve-based model ´ to predict learning curves with unlimited shape and length. Based on the proposed dataset, we modify existing AutoML algorithms to consider the search energy consumption, and our experiments show that the modified energy-aware AutoML methods achieve a better trade-off between energy consumption and model performance

论文信息：

[1] Shuguang Dou, Xinyang Jiang, Cairong Zhao, Dongsheng Li , “EA-HAS-Bench: Energy-aware Hyperparameter and Architecture Search Benchmark,” in Proceeding of Eleventh International Conference on Learning Representations (ICLR 2023，Spotlight), Kigali Convention Center, Virtual-only, May, 2023.

论文链接：

[https://openreview.net/forum?id=n-bvaLSCC78]

代码链接：

[https://github.com/microsoft/EA-HAS-Bench]

视频讲者简介：

窦曙光，同济大学电子与信息工程学院博士生。主要研究为安全可信行人再识别和网络架构搜索。

个人主页：

https://shuguang-52.github.io/

同济大学视觉与智能学习实验室主页：

https://vill.tongji.edu.cn/

特别鸣谢本次论文速览主要组织者：

月度轮值AC：叶茫 (武汉大学)