20170111-02 Jun-Yan Zhu:Deep Learning for Visual Synthesis and Manipulation

报告嘉宾2：Jun-Yan Zhu（UC Berkeley）

报告时间：2017年01月11日（星期三）晚上9：00（北京时间）

报告题目：Deep Learning for Visual Synthesis and Manipulation

主持人：程明明（南开大学）

报告摘要：

Realistic image synthesis and manipulation is challenging because it requires generating and modifying the image appearance in a user-controlled way, while preserving the realism of the result. Unless the user has considerable artistic skill, it is easy to "fall off" the manifold of natural images during generation and editing. In this talk, we propose to learn the natural image manifold directly from data using deep neural networks. We then define a class of image generation and editing operations, and constrain their output to lie on that learned manifold at all times. We present three different approaches: (1) Deep discriminative model: we train a discriminative CNN classifier to predict the realism of the generated result, and optimize an image generation pipeline to maximize the predicted realism score; (2) Deep generative model:  we propose to model the natural image manifold directly via a generative adversarial neural network, and constrain the output to be generated by the generative model; (3) Image-to-Image network: we train a network to map user inputs directly to the final results.

参考文献：

[1] Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman and Alexei A. Efros "Generative Visual Manipulation on the Natural Image Manifold" In ECCV 2016.

[2] Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman and Alexei A. Efros "Learning a Discriminative Model for the Perception of Realism in Composite Images" In ICCV 2015

[3] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros "Image-to-Image Translation with Conditional Adversarial Nets" In arxiv 2016

[4] Jun-Yan Zhu, Yong Jae Lee and Alexei A. Efros, "AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections" In SIGGRAPH 2014

报告人简介：

Jun-Yan is a Computer Science Ph.D. student at UC Berkeley. He received his B.E from Tsinghua University in 2012. Jun-Yan is now working on computer graphics and computer vision with Professor Alexei A. Efros. His current research focuses on summarizing, mining and exploring large-scale visual data, with the goal of building a digital bridge between Humans and Big Visual Data. Jun-Yan is currently supported by a Facebook Fellowship. For more details, visit: www.eecs.berkeley.edu/~junyanz/  

特别鸣谢本次Webinar主要组织者：

VOOC责任委员：李策（兰州理工），张天柱（中科院自动化所）

VODB协调理事：白翔（华中科大），董乐（成电）