TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network

In this work, we present the Text Conditioned Auxiliary Classifier Generative Adversarial Network, (TAC-GAN) a text to image Generative Adversarial Network (GAN) for synthesizing images from their text descriptions. Former approaches have tried to condition the generative process on the textual data; but allying it to the usage of class information, known to diversify the generated samples and improve their structural coherence, has not been explored. We trained the presented TAC-GAN model on the Oxford-102 dataset of flowers, and evaluated the discriminability of the generated images with Inception-Score, as well as their diversity using the Multi-Scale Structural Similarity Index (MS-SSIM). Our approach outperforms the state-of-the-art models, i.e., its inception score is 3.45, corresponding to a relative increase of 7.8% compared to the recently introduced StackGan. A comparison of the mean MS-SSIM scores of the training and generated samples per class shows that our approach is able to generate highly diverse images with an average MS-SSIM of 0.14 over all generated classes.

Project Details

Date: Mar 25, 2017

Author: Ayushman Dash, John Cristian Borges Gamboa, Sheraz Ahmed, Muhammad Zeshan Afzal, Marcus Liwicki

Categories: projectimage understandingimage generationgenerative adversarial neural networks

Website: https://arxiv.org/abs/1703.06412

Related Works.

Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classification

TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network

Multilevel Context Representation for Improving Object Recognition

Music Information Retrieval

Transforming Sensor Data to the Image Domain for Deep Learning - an Application to Footstep Detection

About

We believe that creativity comes with freedom and to solve challenging problems in AI we need to have freedom and creative thinking. We provide an unconstrained environment to highly motivated students to do whatever that comes to their minds and explore deep learning.

Social Links

Our Bunker

Rooms 36-(307/309/310),
TU Kaiserslautern,
Paul-Ehrlich-Str. 36,
67663 Kaiserslautern,
Germany