:::: MENU ::::

Deep Learning-Based Large-Scale Automatic Satellite Crosswalk Classification

Posted in July 26, 2017

I’ve got a paper accepted to the IEEE Geoscience and Remote Sensing Letters. In this paper, we show that freely available crowdsourcing data can be used to accurately (97.11%) train robust deep learning models to perform satellite crosswalk classification on a global scale. Check out the GitHub repository.

Abstract

High-resolution satellite imagery has been increasingly used on remote sensing classification problems. One of the main factors is the availability of this kind of data. Despite the high availability, very little effort has been placed on the zebra crossing classification problem. In this letter, crowdsourcing systems are exploited in order to enable the automatic acquisition and annotation of a large-scale satellite imagery database for crosswalks related tasks. Then, this data set is used to train deep-learning-based models in order to accurately classify satellite images that contain or not contain zebra crossings. A novel data set with more than 240,000 images from 3 continents, 9 countries, and more than 20 cities was used in the experiments. The experimental results showed that freely available crowdsourcing data can be used to accurately (97.11%) train robust models to perform crosswalk classification on a global scale.

Short Description

Overview of the Crosswalk Classification System

System architecture. The input is a region (red dashed rectangle) or a set of regions of interest. First, known crosswalk locations (red markers) are retrieved using the OSM. Second, using Google Maps Directions API, paths (blue dashed arrows) between the crosswalk locations are defined. Third, these paths are decoded and the result locations are filtered (only locations within the green area are accepted) in order to decrease the amount of wrongly annotated images. At this point, positive and negative samples can be downloaded from Google Static Maps API. Finally, this large-scale satellite imagery is used to train ConvNets to perform zebra crossing classification.

The system comprises two parts: automatic data acquisition and annotation, and model training and classification. An overview of the proposed method can be seen in Fig. 1. First, the user defines the regions of interest (regions where he wants to download crosswalks). The region of interest is given by the lower-left and the upper-right corners. After that, crosswalk locations within these regions are retrieved from the OpenStreetMap (OSM). Subsequently, using the zebra crossing locations, positive and negative images (i.e., images that contain and do not contain crosswalks on it, respectively) are downloaded using the Google Static Maps API. As the location of the crosswalks are known, the images are automatically annotated. Finally, these automatically acquired and annotate images are used to train a convolutional neural network (ConvNet) from the scratch to perform classification.

Conclusion

In this letter, a scheme for automatic large-scale satellite zebra crossing classification was proposed. The system automatically acquires images of crosswalks and no-crosswalks around the world using the Google Static Maps API, Google Maps Directions API, and OSM. In addition, deep-learning-based models are trained and evaluated using these automatically annotated images. Experiments were performed on this novel data set with 245 768 images from 3 different continents, 9 countries, and more than 20 cities. The experimental results validated the robustness of the proposed system and showed an accuracy of 97.11% on the global experiment.

Acknowledgements

This work was supported in part by UFES, in part by CAPES for the scholarships, and in part by CNPq under Grant 311120/2016-4. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.

Materials

Source-code and pre-trained models available on the GitHub. I’m still updating this repository, but feel free to ask anything.

BibTeX

@article{
berriel2017grsl,
author    = {Rodrigo F. Berriel and André T. Lopes and Alberto F. de Souza and Thiago Oliveira-Santos}, 
journal   = {IEEE Geoscience and Remote Sensing Letters}, 
title     = {{Deep Learning-Based Large-Scale Automatic Satellite Crosswalk Classification}}, 
year      = {2017}, 
note      = {In press},
doi       = {10.1109/LGRS.2017.2719863}, 
ISSN      = {1545-598X},
}