【15-39期VALSE Webinar活动】  报告嘉宾：李扬彦（Stanford University）
报告时间：2015年12月30日（星期三）晚21:00（北京时间）
报告题目：Joint Embeddings of Shapes and Images via CNN Image Purification [Slides]
主持人： 章国锋（浙江大学）
报告摘要：Both 3D models and 2D images contain a wealth of information about everyday objects in our environment. However, it is difficult to semantically link together these two media forms, even when they feature identical or very similar objects. Real-world images are naturally variable in a number of characteristics such as viewpoint, lighting, background elements, and occlusions. This variability makes it challenging to match images with each other, or with 3D shapes. We propose a joint embedding space populated by both 3D shapes and 2D images, where the distance between embedded entities reflects the similarity between the underlying objects represented by the image or 3D model, unaffected by all the aforementioned nuisance factors. This joint embedding space facilitates comparison between entities of either form, and allows for cross-modality retrieval. We construct the embedding space using an all-pairs 3D shape similarity measure, as 3D shapes are more pure and complete than their appearances in images, leading to more robust distance metrics. We then employ a Convolutional Neural Network (CNN) to "purify" images by muting the distracting factors. The CNN is trained to map an image to a point within the embedding space, such that it is close to a point attributed to a 3D model of a similar object to the one depicted in the image. This purifying capability of the CNN is accomplished with the help of a large amount of training data consisting of images synthesized from 3D shapes. Our deep embedding brings 3D shapes and 2D images into a joint embedding space, where cross-view image retrieval, image-based shape retrieval, as well as shape-based image retrieval tasks are all naturally supported. We evaluate our method on these retrieval tasks and show that it consistently out-performs state-of-the-art methods. Additionally, we demonstrate the usability of a joint embedding in a number of computer graphics applications.
参考文献：
[1] Yangyan Li*, Hao Su*, Charles R. Qi, Noa Fish, Daniel Cohen-Or, and Leonidas Guibas, "Joint Embeddings of Shapes and Images via CNN Image Purification", ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 2015. (*Joint first authors).
[2] Hao Su*, Charles R. Qi*, Yangyan Li, and Leonidas Guibas. "Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views", IEEE International Conference on Computer Vision (ICCV), 2015. (*Joint first authors).
报告人简介：
李扬彦，2013年获中国科学院深圳先进技术研究院博士学位，现为斯坦福大学几何计算研究组（Geometric Computation Group）博士后学者（Postdoctoral Scholar）。其研究兴趣涉及计算机图形学和计算机视觉领域，主要集中于三维数据处理方向。
| 
