Transforming Sensor Data to the Image Domain for Deep Learning - an Application to Footstep Detection

Convolutional Neural Networks (CNNs) have become the state-of-the-art in various computer vision tasks, but they are still premature for most sensor data, especially in pervasive and wearable computing. A major reason for this is the limited amount of annotated training data. In this paper, we propose the idea of leveraging the discriminative power of pre-trained deep CNNs on 2-dimensional sensor data by transforming the sensor modality to the visual domain. By three proposed strategies, 2D sensor output is converted into pressure distribution imageries. Then we utilize a pre-trained CNN for transfer learning on the converted imagery data. We evaluate our method on a gait dataset of floor surface pressure mapping. We obtain a classification accuracy of 87.66%, which outperforms the conventional machine learning methods by over 10%.

Project Details

Date: Jan 4, 2017

Author: Monit Shah Singh, Vinaychandran Pondenkandath, Bo Zhou, Paul Lukowicz, Marcus Liwicki

Categories: projectfootstep detectiontransfer learning


Related Works.

Bidirectional Learning for Robust Neural Networks

Recognizing Challenging Handwritten Annotations with Fully Convolutional Networks

Real-Time Document Image Classification using Deep CNN and Extreme Learning Machines

Cutting the Error by Half: Investigation of Very Deep CNN and Advanced Training Strategies for Document Image Classification

TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network


We believe that creativity comes with freedom and to solve challenging problems in AI we need to have freedom and creative thinking. We provide an unconstrained environment to highly motivated students to do whatever that comes to their minds and explore deep learning.

Social Links

Our Bunker

Building 86,
Davenport 11,
67663 Kaiserslautern,