Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 81 additions & 94 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,7 @@
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/efficient-deep-learning-models-for-land-cover/multi-label-image-classification-on)](https://paperswithcode.com/sota/multi-label-image-classification-on?p=efficient-deep-learning-models-for-land-cover)


# Benchmarking and scaling of deep learning models for land cover image classification.

# Benchmarking and Scaling of Deep Learning Models for Land Cover Image Classification
Code and models from the paper [Benchmarking and scaling of deep learning models for land cover image classification](https://www.sciencedirect.com/science/article/pii/S0924271622003057).

## Citation

## Citation
If you use the models or code provided in this repo, please cite our paper:
```
@article{PAPOUTSIS2023250,
Expand All @@ -24,90 +19,85 @@ abstract = {The availability of the sheer volume of Copernicus Sentinel-2 imager
}
```



## Available pretrained models (All pretrained models can be found [here](https://www.dropbox.com/sh/8bzc1k1e2ejcb7f/AABLwOp_acItoOD7fubevlFKa?dl=0)):
- ### Standard architectures
- #### CNNs
- [DenseNet121](https://www.dropbox.com/s/idenhh7g4j3vapb/checkpoint_densenet121.zip?dl=0)
- [DenseNet169](https://www.dropbox.com/s/qh6cnryod7uric7/checkpoint_DenseNet169.zip?dl=0)
- [DenseNet201](https://www.dropbox.com/s/w0jdbzdamxfyti7/checkpoint_densenet201.zip?dl=0)
- [ResNet50](https://www.dropbox.com/s/trsojsbjvlpt9jz/checkpoint_resnet50.zip?dl=0)
- [ResNet101](https://www.dropbox.com/s/sn2s5xtqlq9721j/checkpoint_resnet101.zip?dl=0)
- [ResNet152](https://www.dropbox.com/s/ukfsg9s8jxjdb64/checkpoint_resnet152.zip?dl=0)
- [VGG16](https://www.dropbox.com/s/mxx7jm6pleb9ezu/checkpoint_vgg16.zip?dl=0)
- [VGG19](https://www.dropbox.com/s/u86oos6i6vp0981/checkpoint_vgg19.zip?dl=0)
- #### Vision Transformers
- [ViT/6](https://www.dropbox.com/s/e7ae92x5vitp12e/checkpoint_vit_6.zip?dl=0)
- [ViT/12](https://www.dropbox.com/s/awyw5uz59cu7rh4/checkpoint_vit_12.zip?dl=0)
- [ViT/20](https://www.dropbox.com/s/nkmqvulbb9wbgdw/checkpoint_vit_20.zip?dl=0)
- [ViT/30](https://www.dropbox.com/s/uaayeo2ep7mif3p/checkpoint_vit_30.zip?dl=0)
- [ViT/40](https://www.dropbox.com/s/yyh8dqngk7j52cw/checkpoint_vit_40.zip?dl=0)
- [ViTM/20](https://www.dropbox.com/s/4gtoqfjyar27g69/checkpoint_vit_m20.zip?dl=0)
- #### MLP Mixer
- [MLPMixer](https://www.dropbox.com/s/29lnfpwc0skkirl/checkpoint_MLPMixer.zip?dl=0)
- [MLPMixer_Tiny](https://www.dropbox.com/s/lqs69eo512xkkt7/checkpoint_MLPMixer_Tiny.zip?dl=0)

- ### Wide-ResNet based Architectures
- [WideResNet](https://www.dropbox.com/s/g2dah9t9zpk5eda/checkpoint_WideResNet.zip?dl=0)
- [WideResNet-COORD](https://www.dropbox.com/s/qf9n1i0a48uinez/checkpoint_WideResNet_COORD.zip?dl=0)
- [WideResNet-CBAM](https://www.dropbox.com/s/ddif8bzfoohx2ap/checkpoint_WideResNet_CBAM.zip?dl=0)
- [WideResNet-CBAM-Ghost](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0)
- [WideResNet-COORD-Ghost](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0)
- [WideResNet-SE](https://www.dropbox.com/s/9c6wykn88ettxfs/checkpoint_WideResNet_SE.zip?dl=0)
- [WideResNet-SE-Ghost](https://www.dropbox.com/s/9azmeuhfy1ukb52/checkpoint_WideResNet_SE_GHOST.zip?dl=0)
- [WideResNet-ECA](https://www.dropbox.com/s/9hqv5xibnx719z0/checkpoint_WideResNet_ECA.zip?dl=0)
- [WideResNet-ECA-GHOST](https://www.dropbox.com/s/51kzqsrdfhby26v/checkpoint_WideResNet_ECA_GHOST.zip?dl=0)


- ### EfficientNet Family
- #### Traditional EfficientNet Architecture
- [EfficientNetB0](https://www.dropbox.com/s/bex8c96r29c9c47/checkpoint_EfficientNetB0.zip?dl=0)
- [EfficientNetB1](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0)
- [EfficientNetB2](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0)
- [EfficientNetB3](https://www.dropbox.com/s/onmted8umdqv9k9/checkpoint_EfficientNetB3.zip?dl=0)
- [EfficientNetB4](https://www.dropbox.com/s/jtvncero96unys0/checkpoint_EfficientNetB4.zip?dl=0)
- [EfficientNetB5](https://www.dropbox.com/s/yrj930z9y8ecogc/checkpoint_EfficientNetB5.zip?dl=0)
- [EfficientNetB6](https://www.dropbox.com/s/qe3mtzrsv6zb4qx/checkpoint_EfficientNetB6.zip?dl=0)
- [EfficientNetB7](https://www.dropbox.com/s/3x6q4869d2t537f/checkpoint_EfficientNetB7.zip?dl=0)

- ### Augmented EfficientNet
- [EfficientNet-CBAM](https://www.dropbox.com/s/nmd8agsr48fhzox/checkpoint_EfficientNet_CBAM.zip?dl=0)
- [EfficientNet-CBAM-GHOST](https://www.dropbox.com/s/aqgn71agf9u44nx/checkpoint_EfficientNet_CBAM_GHOST.zip?dl=0)
- [EfficientNet-COORD](https://www.dropbox.com/s/3i1rjwttsxfpqf3/checkpoint_EfficientNet_COORD.zip?dl=0)
- [EfficientNet-COORD-GHOST](https://www.dropbox.com/s/5lp6ro9ewfcz0iq/checkpoint_EfficientNet_COORD_GHOST.zip?dl=0)
- [EfficientNet-SE](https://www.dropbox.com/s/ubv3ke7qerm2nro/checkpoint_EfficientNet_SE.zip?dl=0)
- [EfficientNet-SE-GHOST](https://www.dropbox.com/s/chagaeszrpgeg44/checkpoint_EfficientNet_SE_GHOST.zip?dl=0)
- [EfficientNet-ECA](https://www.dropbox.com/s/fnhvuw2uze35tu4/checkpoint_EfficientNet_ECA.zip?dl=0)
- [EfficientNet-ECA-GHOST](https://www.dropbox.com/s/3we70krnj06r00a/checkpoint_EfficientNet_ECA_GHOST.zip?dl=0)

- #### ECA EfficientNet Architectures
- [EfficientNetB1-ECA](https://www.dropbox.com/s/6p1yqen3at9le2w/checkpoint_EfficientNet_ECA_B1.zip?dl=0)
- [EfficientNetB2-ECA](https://www.dropbox.com/s/otfz71k9jpg93d7/checkpoint_EfficientNet_ECA_B2.zip?dl=0)
- [EfficientNetB3-ECA](https://www.dropbox.com/s/ncgls9ze7m9v0gn/checkpoint_EfficientNet_ECA_B3.zip?dl=0)
- [EfficientNetB4-ECA](https://www.dropbox.com/s/sn5pzw1eryzsid2/checkpoint_EfficientNet_ECA_B4.zip?dl=0)
- [EfficientNetB5-ECA](https://www.dropbox.com/s/piwgfovu9bwh91t/checkpoint_EfficientNet_ECA_B5.zip?dl=0)
- [EfficientNetB6-ECA](https://www.dropbox.com/s/ryc9hzggwyzmc7k/checkpoint_EfficientNet_ECA_B6.zip?dl=0)
- [EfficientNetB7-ECA](https://www.dropbox.com/s/s1a0c2363eh2j9t/checkpoint_EfficientNet_ECA_B7.zip?dl=0)

- #### Wide-ResNet-ECA based EfficientNet Architectures:
- [WideResNet-ECA-B1](https://www.dropbox.com/s/pmril11fifyy299/checkpoint_WideResNet_ECA_B1.zip?dl=0)
- [WideResNet-ECA-B2](https://www.dropbox.com/s/j0n9jzcwoh0u8t6/checkpoint_WideResNet_ECA_B2.zip?dl=0)
- [WideResNet-ECA-B3](https://www.dropbox.com/s/w18ay6z05t79o9w/checkpoint_WideResNet_ECA_B3.zip?dl=0)
- [WideResNet-ECA-B4](https://www.dropbox.com/s/tx05ebl3een4ye7/checkpoint_WideResNet_ECA_B4.zip?dl=0)
- [WideResNet-ECA-B5](https://www.dropbox.com/s/ohnn76kp2byyttc/checkpoint_WideResNet_ECA_B5.zip?dl=0)
- [WideResNet-ECA-B6](https://www.dropbox.com/s/gdtudop6dim55c5/checkpoint_WideResNet_ECA_B6.zip?dl=0)
- [WideResNet-ECA-B7](https://www.dropbox.com/s/zh7gnkpvicxulto/checkpoint_WideResNet_ECA_B7.zip?dl=0)


## Requirements :

``` tensorflow==2.4.1 ```, ``` horovod==0.21.0 ```

## Usage:
## Available Pretrained Models
All pretrained models can be found [here](https://www.dropbox.com/sh/8bzc1k1e2ejcb7f/AABLwOp_acItoOD7fubevlFKa?dl=0).

### Standard Architectures
| Architecture | Model Name | Link | F-Score |
| :--- | :--- | :--- | :--- |
| **CNNs** | DenseNet121 | [Download](https://www.dropbox.com/s/idenhh7g4j3vapb/checkpoint_densenet121.zip?dl=0) | 76.8% |
| | DenseNet169 | [Download](https://www.dropbox.com/s/qh6cnryod7uric7/checkpoint_DenseNet169.zip?dl=0) | 75.9% |
| | DenseNet201 | [Download](https://www.dropbox.com/s/w0jdbzdamxfyti7/checkpoint_densenet201.zip?dl=0) | 75.6% |
| | ResNet50 | [Download](https://www.dropbox.com/s/trsojsbjvlpt9jz/checkpoint_resnet50.zip?dl=0) | 76.8% |
| | ResNet101 | [Download](https://www.dropbox.com/s/sn2s5xtqlq9721j/checkpoint_resnet101.zip?dl=0) | 76.3% |
| | ResNet152 | [Download](https://www.dropbox.com/s/ukfsg9s8jxjdb64/checkpoint_resnet152.zip?dl=0) | 75.6% |
| | VGG16 | [Download](https://www.dropbox.com/s/mxx7jm6pleb9ezu/checkpoint_vgg16.zip?dl=0) | 77.3% |
| | VGG19 | [Download](https://www.dropbox.com/s/u86oos6i6vp0981/checkpoint_vgg19.zip?dl=0) | 77.7% |
| **Vision Transformers**| ViT/6 | [Download](https://www.dropbox.com/s/e7ae92x5vitp12e/checkpoint_vit_6.zip?dl=0) | 70.2% |
| | ViT/12 | [Download](https://www.dropbox.com/s/awyw5uz59cu7rh4/checkpoint_vit_12.zip?dl=0) | 76.0% |
| | ViT/20 | [Download](https://www.dropbox.com/s/nkmqvulbb9wbgdw/checkpoint_vit_20.zip?dl=0) | 76.6% |
| | ViT/30 | [Download](https://www.dropbox.com/s/uaayeo2ep7mif3p/checkpoint_vit_30.zip?dl=0) | 76.2% |
| | ViT/40 | [Download](https://www.dropbox.com/s/yyh8dqngk7j52cw/checkpoint_vit_40.zip?dl=0) | 75.3% |
| | ViTM/20 | [Download](https://www.dropbox.com/s/4gtoqfjyar27g69/checkpoint_vit_m20.zip?dl=0) | 77.1% |
| **MLP Mixer** | MLPMixer | [Download](https://www.dropbox.com/s/29lnfpwc0skkirl/checkpoint_MLPMixer.zip?dl=0) | 75.2% |
| | MLPMixer_Tiny | [Download](https://www.dropbox.com/s/lqs69eo512xkkt7/checkpoint_MLPMixer_Tiny.zip?dl=0) | 71.6% |

### Wide-ResNet Based Architectures
| Model Name | Link |
| :--- | :--- |
| WideResNet | [Download](https://www.dropbox.com/s/g2dah9t9zpk5eda/checkpoint_WideResNet.zip?dl=0) |
| WideResNet-COORD | [Download](https://www.dropbox.com/s/qf9n1i0a48uinez/checkpoint_WideResNet_COORD.zip?dl=0) |
| WideResNet-CBAM | [Download](https://www.dropbox.com/s/ddif8bzfoohx2ap/checkpoint_WideResNet_CBAM.zip?dl=0) |
| WideResNet-CBAM-Ghost | [Download](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0) |
| WideResNet-COORD-Ghost | Link Broken |
| WideResNet-SE | [Download](https://www.dropbox.com/s/9c6wykn88ettxfs/checkpoint_WideResNet_SE.zip?dl=0) |
| WideResNet-SE-Ghost | [Download](https://www.dropbox.com/s/9azmeuhfy1ukb52/checkpoint_WideResNet_SE_GHOST.zip?dl=0) |
| WideResNet-ECA | [Download](https://www.dropbox.com/s/9hqv5xibnx719z0/checkpoint_WideResNet_ECA.zip?dl=0) |
| WideResNet-ECA-GHOST | [Download](https://www.dropbox.com/s/51kzqsrdfhby26v/checkpoint_WideResNet_ECA_GHOST.zip?dl=0) |

### EfficientNet Family
| Architecture | Model Name | Link |
| :--- | :--- | :--- |
| **Traditional EfficientNet** | EfficientNetB0 | [Download](https://www.dropbox.com/s/bex8c96r29c9c47/checkpoint_EfficientNetB0.zip?dl=0) |
| | EfficientNetB1 | [Download](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) |
| | EfficientNetB2 | [Download](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) |
| | EfficientNetB3 | [Download](https://www.dropbox.com/s/onmted8umdqv9k9/checkpoint_EfficientNetB3.zip?dl=0) |
| | EfficientNetB4 | [Download](https://www.dropbox.com/s/jtvncero96unys0/checkpoint_EfficientNetB4.zip?dl=0) |
| | EfficientNetB5 | [Download](https://www.dropbox.com/s/yrj930z9y8ecogc/checkpoint_EfficientNetB5.zip?dl=0) |
| | EfficientNetB6 | [Download](https://www.dropbox.com/s/qe3mtzrsv6zb4qx/checkpoint_EfficientNetB6.zip?dl=0) |
| | EfficientNetB7 | [Download](https://www.dropbox.com/s/3x6q4869d2t537f/checkpoint_EfficientNetB7.zip?dl=0) |
| **Augmented EfficientNet**| EfficientNet-CBAM | [Download](https://www.dropbox.com/s/nmd8agsr48fhzox/checkpoint_EfficientNet_CBAM.zip?dl=0) | 76.1% |
| | EfficientNet-CBAM-GHOST | [Download](https://www.dropbox.com/s/aqgn71agf9u44nx/checkpoint_EfficientNet_CBAM_GHOST.zip?dl=0) |
| | EfficientNet-COORD | [Download](https://www.dropbox.com/s/3i1rjwttsxfpqf3/checkpoint_EfficientNet_COORD.zip?dl=0) |
| | EfficientNet-COORD-GHOST| [Download](https://www.dropbox.com/s/5lp6ro9ewfcz0iq/checkpoint_EfficientNet_COORD_GHOST.zip?dl=0) |
| | EfficientNet-SE | [Download](https://www.dropbox.com/s/ubv3ke7qerm2nro/checkpoint_EfficientNet_SE.zip?dl=0) |
| | EfficientNet-SE-GHOST | [Download](https://www.dropbox.com/s/chagaeszrpgeg44/checkpoint_EfficientNet_SE_GHOST.zip?dl=0) |
| | EfficientNet-ECA | [Download](https://www.dropbox.com/s/fnhvuw2uze35tu4/checkpoint_EfficientNet_ECA.zip?dl=0) |
| | EfficientNet-ECA-GHOST | [Download](https://www.dropbox.com/s/3we70krnj06r00a/checkpoint_EfficientNet_ECA_GHOST.zip?dl=0) |
| **ECA EfficientNet** | EfficientNetB1-ECA | [Download](https://www.dropbox.com/s/6p1yqen3at9le2w/checkpoint_EfficientNet_ECA_B1.zip?dl=0) |
| | EfficientNetB2-ECA | [Download](https://www.dropbox.com/s/otfz71k9jpg93d7/checkpoint_EfficientNet_ECA_B2.zip?dl=0) |
| | EfficientNetB3-ECA | [Download](https://www.dropbox.com/s/ncgls9ze7m9v0gn/checkpoint_EfficientNet_ECA_B3.zip?dl=0) |
| | EfficientNetB4-ECA | [Download](https://www.dropbox.com/s/sn5pzw1eryzsid2/checkpoint_EfficientNet_ECA_B4.zip?dl=0) |
| | EfficientNetB5-ECA | [Download](https://www.dropbox.com/s/piwgfovu9bwh91t/checkpoint_EfficientNet_ECA_B5.zip?dl=0) |
| | EfficientNetB6-ECA | [Download](https://www.dropbox.com/s/ryc9hzggwyzmc7k/checkpoint_EfficientNet_ECA_B6.zip?dl=0) |
| | EfficientNetB7-ECA | [Download](https://www.dropbox.com/s/s1a0c2363eh2j9t/checkpoint_EfficientNet_ECA_B7.zip?dl=0) |
| **Wide-ResNet-ECA** | WideResNet-ECA-B1 | [Download](https://www.dropbox.com/s/pmril11fifyy299/checkpoint_WideResNet_ECA_B1.zip?dl=0) |
| | WideResNet-ECA-B2 | [Download](https://www.dropbox.com/s/j0n9jzcwoh0u8t6/checkpoint_WideResNet_ECA_B2.zip?dl=0) |
| | WideResNet-ECA-B3 | [Download](https://www.dropbox.com/s/w18ay6z05t79o9w/checkpoint_WideResNet_ECA_B3.zip?dl=0) |
| | WideResNet-ECA-B4 | [Download](https://www.dropbox.com/s/tx05ebl3een4ye7/checkpoint_WideResNet_ECA_B4.zip?dl=0) |
| | WideResNet-ECA-B5 | [Download](https://www.dropbox.com/s/ohnn76kp2byyttc/checkpoint_WideResNet_ECA_B5.zip?dl=0) |
| | WideResNet-ECA-B6 | [Download](https://www.dropbox.com/s/gdtudop6dim55c5/checkpoint_WideResNet_ECA_B6.zip?dl=0) |
| | WideResNet-ECA-B7 | [Download](https://www.dropbox.com/s/zh7gnkpvicxulto/checkpoint_WideResNet_ECA_B7.zip?dl=0) |

## Dataset
The dataset used is the BigEarthNet 19 class nomenclature instead of the original 43 class nomenclature.

## Requirements
`pip install -r .\requirements.txt`

## Usage
To run an experiment modify the [config file](configs/base.json) and execute train.py. Example for MLPMixer with batch size = 100 and learning rate 1e-4:
```

{
"model_name": "MLPMixer",
"hparams": {"phi": 1.0, "alpha": 1.0, "beta": 1.0, "gamma": 1.0, "dropout": 0.1},
Expand All @@ -133,17 +123,14 @@ abstract = {The availability of the sheer volume of Copernicus Sentinel-2 imager
"eval_checkpoint": "/work2/pa20/ipapout/gitSpace/TF1.10.1gpu_Py3/NikosTmp/v2/charmbigearth/bigearthnet-tf2/bestTestResNet50/checkpoint_ResNet50",
"augment": true
}

```

To execute in a single-GPU machine:
```
python3 train.py --parallel=False
```

or for multi node training :
or for multi node training:
```
horovodrun --gloo -np $SLURM_NTASKS -H $WORKERS --network-interface ib0 --start-timeout 120 --gloo-timeout-seconds 120 python3 train.py --parallel=True
```


```