20161214-40梁小丹：Deep Variation-structured Reinforcement Learning for...

报告嘉宾2：梁小丹（CMU）

报告时间：2016年12月14日（星期三）晚上9：00（北京时间）

报告题目：Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection

主持人：左旺孟（哈尔滨工业大学）

报告摘要：

Despite progress in visual perception tasks such as image classification and detection, computers still struggle to understand the interdependency of objects in the scene as a whole, e.g., relations between objects or their attributes. Existing methods often ignore global context cues capturing the interactions among different object instances, and can only recognize a handful of types by exhaustively training individual detectors for all possible relationships. To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image. First, a directed semantic action graph is built using language priors to provide a richand compact representation of semantic correlations between object categories, predicates, and attributes. Next,we use a variation-structured traversal over the action graph to construct a small, adaptive action set for each step based on the current state and historical actions. In particular, an ambiguity-aware object mining scheme is used to resolve semantic ambiguity among object categories that the object detector fails to distinguish. We then make sequential predictions using a deep RL framework, incorporating global context cues and semantic embedding of previously extracted phrases in the state vector. Our experiments on the Visual Relationship Detection (VRD) dataset and the large-scale Visual Genome dataset validate the superiority of VRL, which can achieve significantly better detection results on datasets involving thousands of relationship and attribute types.

报告人简介：

Xiaodan Liang is currently a postdoctoral research fellow at the Machine Learning Department, Carnegie Mellon University, working with Prof. Eric P. Xing.She obtained my Ph.D. degree in the School of Data and Computer Science at Sun Yat-sen University in June 2016, advised by Prof. Liang Lin. She was a visiting scholar from March, 2014 to March, 2016 in the Department of EECS of the National University of Singapore, working with Prof. Shuicheng Yan. She have closely collaborated with Dr. Xiaohui Shen in Adobe Research and Dr. Jianchao Yang in Snapchat Research from 2014 to 2016. 

特别鸣谢本次Webinar主要组织者：

VOOC责任委员：邓伟洪（北京邮电大学）

VODB协调理事：林倞（中山大学），郑伟诗（中山大学）