20160413-11 于茜: Sketch Me that Shoe

报告时间：2016年4月13日（周三）晚20：00（北京时间）

报告题目：Sketch Me that Shoe

主持人：杨恒

报告摘要：

We investigate the problem of fine-grained sketch-based image retrieval (SBIR), where free-hand human sketches are used as queries to perform instance-level retrieval of images. This is an extremely challenging task because (i) visual comparisons not only need to be fine-grained but also executed cross-domain, (ii) free-hand (finger) sketches are highly abstract, making fine-grained matching harder, and most importantly (iii) annotated cross-domain sketch-photo datasets required for training are scarce, challenging many state-of-the-art machine learning techniques.

In this paper, for the first time, we address all these challenges, providing a step towards the capabilities that would underpin a commercial sketch-based image retrieval application. We introduce a new database of 1,432 sketch- photo pairs from two categories with 32,000 fine-grained triplet ranking annotations. We then develop a deep triplet- ranking model for instance-level SBIR with a novel data augmentation and staged pre-training strategy to alleviate the issue of insufficient fine-grained training data. Extensive experiments are carried out to contribute a variety of insights into the challenges of data sufficiency and over-fitting avoidance when training deep networks for fine-grained cross-domain ranking tasks.

参考文献：

[1] Q. Yu, F. Liu, Y. Song, T. Xiang, and T. Hospedales, C. C. Loy. Sketch Me that Shoe. In CVPR, 2016. (Oral)

[2] Q. Yu, Y. Yang, Y. Song, T. Xiang, and T. Hospedales. Sketch-a-net that beats humans. In BMVC, 2015. (Best science paper)

报告人简介：

Qian Yu is a second year Ph.D. student in Computer Vision group at Queen Mary University of London, advised by Dr. Tao Xiang and Dr. Yi-Zhe Song. Her research focuses on sketch recognition and sketch-based image retrieval. She is also interested in machine learning and human computer interaction.