Bidirectional Learning for Robust Neural Networks

A multilayer perceptron can behave as a generative classifier by applying bidirectional learning (BL). It consists of training an undirected neural network to map input to output and vice-versa; therefore it can produce a classifier in one direction, and a generator in the opposite direction for the same data. In this paper, two novel learning techniques are introduced which use BL for improving robustness to white noise static and adversarial examples. The first method is bidirectional propagation of errors, which the error propagation occurs in backward and forward directions. Motivated by the fact that its generative model receives as input a constant vector per class, we introduce as a second method the hybrid adversarial networks (HAN). Its generative model receives a random vector as input and its training is based on generative adversarial networks (GAN). To assess the performance of BL, we perform experiments using several architectures with fully and convolutional layers, with and without bias. Experimental results show that both methods improve robustness to white noise static and adversarial examples, but have different behaviour depending on the architecture and task, being more beneficial to use the one or the other. Nevertheless, HAN using a convolutional architecture with batch normalization presents outstanding robustness, reaching state-of-the-art accuracy on adversarial examples of hand-written digits.

Project Details

Date: May 22, 2018

Author: Sidney Pontes-Filho, Marcus Liwicki

Categories: projectGANsadversarialdeep learning


Related Works.

Bidirectional Learning for Robust Neural Networks

Recognizing Challenging Handwritten Annotations with Fully Convolutional Networks

Real-Time Document Image Classification using Deep CNN and Extreme Learning Machines

Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classification

TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network


We believe that creativity comes with freedom and to solve challenging problems in AI we need to have freedom and creative thinking. We provide an unconstrained environment to highly motivated students to do whatever that comes to their minds and explore deep learning.

Social Links

Our Bunker

Building 86,
Davenport 11,
67663 Kaiserslautern,