metadata

title: >-
  Satellite extractor: Towards a Smart  Eco-epidemiological Model of Dengue in
  Colombia  using Satellite Imagery*.
emoji: 🌍
colorFrom: yellow
colorTo: green
sdk: static
pinned: false

Sentinelhub grant: Sponsoring request ID 1c081a - Towards a Smart Eco-epidemiological Model of Dengue in Colombia using Satellite in Collaboration with MIT Critical Data Colombia.

Project supported by ESA Network of Resources Initiative

Project proposal (READ MORE)

Project Organisation: MIT Critical Data Colombia.

Model Description

Repository: Code
The proposed dataset format is shared in Metadengue
ESA project: Sponsoring request ID 1c081a - Towards a Smart Eco-epidemiological Model of Dengue in Colombia using Satellite in Collaboration with MIT Critical Data Colombia)
Point of Contact: Sebastian A. Cajas Ordóñez

Summary

Here below find all the dataset's versions and descriptions.

Baseline method from satellite extractor: Raw data from Sentinel 2LC1 with recursive artifact removal, clouds removal based on LeastCC and Nearest Interpolation for spatial resolution.

SAT1_dataset_5_best_cities: Top 5 municipalities based on Baseline method from satellite extractor
SAT2_dataset_10_best_cities: Top 10 municipalities based on Baseline method from satellite extractor
SAT3_FULL_COLOMBIA: Top 81 municipalities based on Baseline method from satellite extractor
SAT4_dataset_10_best_cities_augmented_v1: Augmented data with aligned metadata. Data was extracted using recursive artifact removal, cloud removal based on LeastCC, and Nearest Interpolation for spatial resolution. Implemented here and augmentations applied to RGB channels while leaving other satellite channels unchanged:

Pre-processing: The first step is to apply Contrast Limited Adaptive Histogram Equalization (CLAHE) to the image, with a clip limit of 6.0 and a tile grid size of 16 by 16. This technique enhances the contrast of the image while preventing over-amplification of noise. Secondly, we apply the RGBShift augmentation technique, which randomly shifts the values of pixels in the red, green, and blue channels of the image. This is done with a probability of 100% and is applied to 30 pixels per channel. Finally, we apply the RandomBrightnessContrast augmentation technique with a probability of 50%. This technique randomly adjusts the brightness and contrast of the image to create variations in the dataset.
SAT5_dataset_10_best_cities_augmented_v2: These images are improved to remove near black images method using a recursive forward-backward artefact removal algorithm with inter-band data augmentation on satellite imagery. Augmented data with aligned metadata. Improved version using Albumentation wrapper modules with extra augmented data. Data extracted using recursive artifact removal, cloud removal based on LeastCC, and Nearest Interpolation for spatial resolution. Implemented Notebook and augmentations applied to RGB channels while leaving other satellite channels unchanged.

Pre-processing: The image augmentation techniques used in this process include various forms of Gaussian noise, implemented through IAAAdditiveGaussianNoise with a probability of 20% and mean zero, and standard deviation of 0.01 * 255 or 0.05 * 255. The GaussNoise technique was also employed with a mean of zero and default variance of (10.0, 50.0). General blurring was implemented through MotionBlur (p=.2), MedianBlur (blur_limit=3, p=0.1), Blur(blur_limit=3, p=0.1), and ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.2, rotate_limit=45, p=0.2). Distortion techniques included OpticalDistortion(p=0.3), GridDistortion(p=.1), IAAPiecewiseAffine(p=0.3), and IAAAffine(scale=(0.8, 1.2), translate_percent=0.1, rotate=15, shear=10, p=0.2). Finally, brightness adjustments were made using CLAHE(clip_limit=2), IAASharpen(), IAAEmboss(), RandomBrightnessContrast(), and HueSaturationValue(p=0.3).
Creating Cloud-Cloudless Paired Dataset: This dataset, derived from imagery in five Colombian municipalities, consists of 1640 images (820 pairs), where each of the 164 images per municipality is paired with a previously identified optimal cloudless image. The Cloud2CloudlesDataset class organizes these pairs into a new folder (DATASET), with images renamed to indicate ground truth and cloud presence. The class, initialized with source and destination paths, includes tests for image count verification and folder existence confirmation.

Reading data

The data can be read as (example):

from datasets import load_dataset

dataset = load_dataset("MITCriticalData/Unlabeled_top_10_cities_forward_backward_alg")

Alternatively, for datasets<1.11.0 the lecture of .tiff and .json files is not compatible. In such case we recommend to download the data as:

wget path_to_data/images.zip && unzip images.zip -d images/

wget path_to_data/annotations.zip && unzip annotations.zip -d annotations/

Licensing Information

The dataset is released under the terms of MIT. By using this, you are also bound to the respective Terms of Use and License of the original source.

Author & Mantainer

Sebastián Cajas Ordóñez

Contributors

MIT Critical data Colombia Team: Sebastian A. Cajas, David Restrepo, Kuan-Ting Kuo, Dana Moukheiber, Atika Rahman Paddo, Saptarshi Purkayastha, Leo Anthony Celi, Po-Chih Kuo, Juan Sebastián Osorio-Valencia, Kuan-Ting Ku, Braiam Escobar, Diego M. López, Cheng Che Tsai, Wilson Arbey Diaz, Luis Jesús Martínez, Alessa Álvarez, Siyi Tang, Amara Tariq, Imon Banerjee, Aakanksha Rana, Maria Patricia Arbelaez-Montoy, Cheng Che Tsai, Laura Sofía Daza Rosero, Jhon Fredy Romero Núñez, Wilson Arbey Diaz, Luis Jesús Martínez, Saketh Sundar, Alessa Álvarez, Siyi Tang, Amara Tariq, Imon Banerjee, Aakanksha Rana, Ivan Darío Velez, Maria Patricia Arbelaez-Montoya.

Citation

Please cite our work if you find the resources in this repository useful:

Sebastian Andres Cajas Ordoñez, David Restrepo, Kuan-Ting Kuo, Dana Moukheiber, Atika Rahman Paddo, Leo Anthony Celi, Po-Chih Kuo (2022). Towards a Smart Eco-epidemiological Model of Dengue in Colombia using Satellite [Source code]. GitHub. https://github.com/sebasmos/satellite.extractor

Spaces:

MITCriticalData
/

README

Running