Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
1 result
Compare
Name Last commit Last update
README.md
README.md

Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning

This repository contains code of the paper abstract Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning submitted to IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2023. This work has been done at the Remote Sensing Image Analysis group by Baris Buyuktas, Gencer Sumbul, and Begüm Demir.

This repository contains (in parts) code that has been adapted from:

Introduction

Remote sensing (RS) images are usually stored in compressed format to reduce the storage size of the archives. Thus, existing content-based image retrieval (CBIR) systems in RS require decoding images before applying CBIR (which is computationally demanding in the case of large-scale CBIR problems). To address this problem, in this paper, we present a joint framework that simultaneously learns RS image compression and indexing, eliminating the need for decoding RS images before applying CBIR. The proposed framework is made up of two modules. The first module aims at effectively compressing RS images. It is achieved based on an auto-encoder architecture. The second module aims at producing hash codes with a high discrimination capability. It is achieved based on a deep hashing method that exploits soft pairwise, bit-balancing and classification loss functions. We also propose a two stage learning strategy with gradient manipulation techniques to obtain image representations that are compatible with both RS image indexing and compression.

Prerequisites

The code in this repository requires Python 3.10.4, pytorch 1.12.1.

The code is tested in Ubuntu 18.04.

An exemplary setup which contains everything needed:

(1) wget  https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
(2) bash Miniconda3-latest-Linux-x86_64.sh (say yes to append path to bashrc)
(3) source .bashrc
(4) sudo add-apt-repository ppa:ubuntugis/ppa && sudo apt-get update && sudo apt-get install g++
(5) sudo apt-get install libgdal-dev gdal-bin
(6) ogrinfo --version
(7) conda activate base
(8) conda install matplotlib scipy scikit-learn scikit-image tqdm pillow pytorch

Training Arguments

  • exp_id: Id of the experiment
  • suffix: suffix placed in front of the config file name
  • model_name: Model used for training,validation,testing (SimCLR, deepSM, S2MC, DSCMR, DUCH, DCCA, SSCMCBIR)
  • train_tfrecord_paths: TFRecord file(s) for training.
  • val_tfrecord_paths: TFRecord file(s) for validation.
  • test_tfrecord_paths: TFRecord file(s) for testing.
  • batch_size: Batch size used during training.
  • embed_dim: Embedding dimension for projection heads
  • learning_rate: The initial learning rate.
  • optimizer : Optimizers used for training,Adam or SGD
  • nb_epoch: The number of epochs for the training.
  • gpu: gpu(s) used for for training, validation, retrieval
  • dumps:
  • model_weights: directory containing the trained model weights , e.g. ../dumps_paper/weights/,
  • summaries: directory containing the summary files (tf.event) of the training, e.g. ../dumps_paper/summaries/,
  • configs: directory containing the config files of the training, e.g. ../dumps_paper/configs/,
  • logs: directory containing the log files of the training, e.g. ../dumps_paper/logs/,
  • features: directory containing the feature h5 files of the retrieval for a specific model, e.g. ../dumps_paper/features/
  • dataset: Dataset used for training, validation, retrieval (BEN)
  • nb_class: number of classes used in the dataset, 19 for BEN

Authors

Baris Buyuktas https://rsim.berlin/team/members/baris-buyuktas

Gencer Sümbül https://rsim.berlin/team/members/gencer-sumbul

License

The code in this repository is licensed under the MIT License:

MIT License

Copyright (c) 2022 The Authors of The Paper, "Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learning"

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.