S²-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images

This repository contains the code of the paper S²-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images accepted at the IEEE International Geoscience and Remote Sensing Symposium, 2020. This work has been done at the Remote Sensing Image Analysis group by Jose Luis Holgado, Mahdyar Ravanbakhsh and Begüm Demir.

Abstract

Deep Neural Networks have recently demonstrated promising performance in binary change detection (CD) problems in remote sensing (RS), requiring a large amount of labeled multitemporal training samples. Since collecting such data is time-consuming and costly, most of the existing methods rely on pre-trained networks on publicly available computer vision (CV) datasets. However, because of the differences in image characteristics in CV and RS, this approach limits the performance of the existing CD methods. To address this problem, we propose a self-supervised conditional Generative Adversarial Network (S²-cGAN). The proposed S²-cGAN is trained to generate only the distribution of unchanged samples. To this end, the proposed method consists of two main steps: 1) Generating a reconstructed version of the input image as an unchanged image 2) Learning the distribution of unchanged samples through an adversarial game. Unlike the existing GAN based methods (which only use the discriminator during the adversarial training to supervise the generator), the S²-cGAN directly exploits the discriminator likelihood to solve the binary CD task. Experimental results show the effectiveness of the proposed S²-cGAN when compared to the state of the art CD methods.

If you use our code, please cite our paper as:

J. L. Holgado Alvarez, M. Ravanbakhsh, B. Demіr, "S²-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images", IEEE International Geoscience and Remote Sensing Symposium, Hawaii, USA, 2020.

@article{IG203936,
  author={J.L. {Holgado Alvarez}, M. {Ravanbakhsh} and B. {Demіr}},
  journal={IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium}, 
  title={S^2-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images}, 
  year={2020}
}

S²-cGAN Change Detection Strategy

The following illustration represents the change detection strategy introduced in S²-cGAN.

For the details please check our paper paper.

Examples of score maps associated to a pair of patches, and its comparison to the ground-truth.

For the details please check our paper paper.

Setup

Prerequisites

• The code is tested with Python 3.7, PyTorch 0.4.1 and Ubuntu 18.04.
• Please, check out the requirements file for more information.
• For reproducibility, you can find our train and test sets here (password: Igarss_2020).

Hardware setup

The model was trained and tested in a machine with the following features:

• Nvidia Quadro P2000
• 16 GB RAM
• 12 CPU cores

Dataset

We used Worldview 2 satellite Very High Spatial Resolution (VHR) multispectral images dataset provided here.

Construction of Train and Test Sets

 python data_creator.py 

The script data_creator.py expects the following arguments, you can used the default values or pass them as command arguments:

• --band3 directory location of the rgb channels
• --band4 directory location of the I channel
• --output_dir output parent directory
• --test_dir output directory for testset json file
• --train_dir output directory for trainset json file
• --eval_dir output directory for evaluation json file
• --gt name and extension for ground-truth patches
• --A name and extension for T1 patches
• --Bname and extension for T2 patches
• --city identification code template for patches, by default wv2{}_{}. the first gap corresponds to tile id and the second gap correspond to patch id in the tile
• --custom_filter_active boolean variable to trigger filter operations over the trainset, the filter consist in remove patches from the trainset that contains a ratio of change/no_change bigger than certain threshold
• --filter_value, filter threshold value (0, 1)
• --kernel patch size, by default 128
• --offset padding value, by default 0
• --stride stride value, by default 10

Training Phase

 python train.py --dataroot /patches --dataset /train_set/patch_coords.json --name S2_NET_001 

The script train.py expects the following command line arguments:

• --dataroot path to patches folder
• --name network name
• --dataset specify the path to the trainset json file

Testing Phase

 python test.py --dataroot /patches --dataset /test_set/patch_coords.json --name S2_NET_001 --results_dir /results  

The script test.py expects the following command line arguments:

• --dataroot path to patches folder
• --name network name
• --dataset specify the path to the testset json file
• --results_dir specify the path to the output directory

Performance Evaluation

 python performance_evaluation.py  --dataroot /patches --dataset /perfomance_evaluation/perfomance_evaluation_coords.json --results_dir /results

The script performance_evaluation.py expects the following command line arguments:

• --dataroot path to patches folder
• --dataset specify the path to the perfomance_evaluation json file
• --results_dir specify the path to the output directory created by test.py

Authors

Jose Luis Holgado Alvarez

Acknowledgements

This software was developed based on: P. Isola, J. Zhu, T. Zhou, A. A. Efros. Image-to-Image Translation with Conditional Adversarial Nets , 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Maintained by

Jose Luis Holgado Alvarez

E-mail:

• jose.l.holgadoalvarez@campus.tu-berlin.de
• jlhalv92@gmail.com

License

The code in this repository to facilitate the use of the S2-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images is licensed under the MIT License:

MIT License

Copyright (c) 2020 The Authors of The Paper, " S2-cGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images"

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.