diff --git a/README.md b/README.md index 4779560..fd49133 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,7 @@ -[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/efficient-deep-learning-models-for-land-cover/multi-label-image-classification-on)](https://paperswithcode.com/sota/multi-label-image-classification-on?p=efficient-deep-learning-models-for-land-cover) - - -# Benchmarking and scaling of deep learning models for land cover image classification. - +# Benchmarking and Scaling of Deep Learning Models for Land Cover Image Classification Code and models from the paper [Benchmarking and scaling of deep learning models for land cover image classification](https://www.sciencedirect.com/science/article/pii/S0924271622003057). -## Citation - +## Citation If you use the models or code provided in this repo, please cite our paper: ``` @article{PAPOUTSIS2023250, @@ -24,90 +19,85 @@ abstract = {The availability of the sheer volume of Copernicus Sentinel-2 imager } ``` - - -## Available pretrained models (All pretrained models can be found [here](https://www.dropbox.com/sh/8bzc1k1e2ejcb7f/AABLwOp_acItoOD7fubevlFKa?dl=0)): -- ### Standard architectures - - #### CNNs - - [DenseNet121](https://www.dropbox.com/s/idenhh7g4j3vapb/checkpoint_densenet121.zip?dl=0) - - [DenseNet169](https://www.dropbox.com/s/qh6cnryod7uric7/checkpoint_DenseNet169.zip?dl=0) - - [DenseNet201](https://www.dropbox.com/s/w0jdbzdamxfyti7/checkpoint_densenet201.zip?dl=0) - - [ResNet50](https://www.dropbox.com/s/trsojsbjvlpt9jz/checkpoint_resnet50.zip?dl=0) - - [ResNet101](https://www.dropbox.com/s/sn2s5xtqlq9721j/checkpoint_resnet101.zip?dl=0) - - [ResNet152](https://www.dropbox.com/s/ukfsg9s8jxjdb64/checkpoint_resnet152.zip?dl=0) - - [VGG16](https://www.dropbox.com/s/mxx7jm6pleb9ezu/checkpoint_vgg16.zip?dl=0) - - [VGG19](https://www.dropbox.com/s/u86oos6i6vp0981/checkpoint_vgg19.zip?dl=0) - - #### Vision Transformers - - [ViT/6](https://www.dropbox.com/s/e7ae92x5vitp12e/checkpoint_vit_6.zip?dl=0) - - [ViT/12](https://www.dropbox.com/s/awyw5uz59cu7rh4/checkpoint_vit_12.zip?dl=0) - - [ViT/20](https://www.dropbox.com/s/nkmqvulbb9wbgdw/checkpoint_vit_20.zip?dl=0) - - [ViT/30](https://www.dropbox.com/s/uaayeo2ep7mif3p/checkpoint_vit_30.zip?dl=0) - - [ViT/40](https://www.dropbox.com/s/yyh8dqngk7j52cw/checkpoint_vit_40.zip?dl=0) - - [ViTM/20](https://www.dropbox.com/s/4gtoqfjyar27g69/checkpoint_vit_m20.zip?dl=0) - - #### MLP Mixer - - [MLPMixer](https://www.dropbox.com/s/29lnfpwc0skkirl/checkpoint_MLPMixer.zip?dl=0) - - [MLPMixer_Tiny](https://www.dropbox.com/s/lqs69eo512xkkt7/checkpoint_MLPMixer_Tiny.zip?dl=0) - -- ### Wide-ResNet based Architectures - - [WideResNet](https://www.dropbox.com/s/g2dah9t9zpk5eda/checkpoint_WideResNet.zip?dl=0) - - [WideResNet-COORD](https://www.dropbox.com/s/qf9n1i0a48uinez/checkpoint_WideResNet_COORD.zip?dl=0) - - [WideResNet-CBAM](https://www.dropbox.com/s/ddif8bzfoohx2ap/checkpoint_WideResNet_CBAM.zip?dl=0) - - [WideResNet-CBAM-Ghost](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0) - - [WideResNet-COORD-Ghost](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0) - - [WideResNet-SE](https://www.dropbox.com/s/9c6wykn88ettxfs/checkpoint_WideResNet_SE.zip?dl=0) - - [WideResNet-SE-Ghost](https://www.dropbox.com/s/9azmeuhfy1ukb52/checkpoint_WideResNet_SE_GHOST.zip?dl=0) - - [WideResNet-ECA](https://www.dropbox.com/s/9hqv5xibnx719z0/checkpoint_WideResNet_ECA.zip?dl=0) - - [WideResNet-ECA-GHOST](https://www.dropbox.com/s/51kzqsrdfhby26v/checkpoint_WideResNet_ECA_GHOST.zip?dl=0) - - -- ### EfficientNet Family - - #### Traditional EfficientNet Architecture - - [EfficientNetB0](https://www.dropbox.com/s/bex8c96r29c9c47/checkpoint_EfficientNetB0.zip?dl=0) - - [EfficientNetB1](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) - - [EfficientNetB2](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) - - [EfficientNetB3](https://www.dropbox.com/s/onmted8umdqv9k9/checkpoint_EfficientNetB3.zip?dl=0) - - [EfficientNetB4](https://www.dropbox.com/s/jtvncero96unys0/checkpoint_EfficientNetB4.zip?dl=0) - - [EfficientNetB5](https://www.dropbox.com/s/yrj930z9y8ecogc/checkpoint_EfficientNetB5.zip?dl=0) - - [EfficientNetB6](https://www.dropbox.com/s/qe3mtzrsv6zb4qx/checkpoint_EfficientNetB6.zip?dl=0) - - [EfficientNetB7](https://www.dropbox.com/s/3x6q4869d2t537f/checkpoint_EfficientNetB7.zip?dl=0) - - - ### Augmented EfficientNet - - [EfficientNet-CBAM](https://www.dropbox.com/s/nmd8agsr48fhzox/checkpoint_EfficientNet_CBAM.zip?dl=0) - - [EfficientNet-CBAM-GHOST](https://www.dropbox.com/s/aqgn71agf9u44nx/checkpoint_EfficientNet_CBAM_GHOST.zip?dl=0) - - [EfficientNet-COORD](https://www.dropbox.com/s/3i1rjwttsxfpqf3/checkpoint_EfficientNet_COORD.zip?dl=0) - - [EfficientNet-COORD-GHOST](https://www.dropbox.com/s/5lp6ro9ewfcz0iq/checkpoint_EfficientNet_COORD_GHOST.zip?dl=0) - - [EfficientNet-SE](https://www.dropbox.com/s/ubv3ke7qerm2nro/checkpoint_EfficientNet_SE.zip?dl=0) - - [EfficientNet-SE-GHOST](https://www.dropbox.com/s/chagaeszrpgeg44/checkpoint_EfficientNet_SE_GHOST.zip?dl=0) - - [EfficientNet-ECA](https://www.dropbox.com/s/fnhvuw2uze35tu4/checkpoint_EfficientNet_ECA.zip?dl=0) - - [EfficientNet-ECA-GHOST](https://www.dropbox.com/s/3we70krnj06r00a/checkpoint_EfficientNet_ECA_GHOST.zip?dl=0) - - - #### ECA EfficientNet Architectures - - [EfficientNetB1-ECA](https://www.dropbox.com/s/6p1yqen3at9le2w/checkpoint_EfficientNet_ECA_B1.zip?dl=0) - - [EfficientNetB2-ECA](https://www.dropbox.com/s/otfz71k9jpg93d7/checkpoint_EfficientNet_ECA_B2.zip?dl=0) - - [EfficientNetB3-ECA](https://www.dropbox.com/s/ncgls9ze7m9v0gn/checkpoint_EfficientNet_ECA_B3.zip?dl=0) - - [EfficientNetB4-ECA](https://www.dropbox.com/s/sn5pzw1eryzsid2/checkpoint_EfficientNet_ECA_B4.zip?dl=0) - - [EfficientNetB5-ECA](https://www.dropbox.com/s/piwgfovu9bwh91t/checkpoint_EfficientNet_ECA_B5.zip?dl=0) - - [EfficientNetB6-ECA](https://www.dropbox.com/s/ryc9hzggwyzmc7k/checkpoint_EfficientNet_ECA_B6.zip?dl=0) - - [EfficientNetB7-ECA](https://www.dropbox.com/s/s1a0c2363eh2j9t/checkpoint_EfficientNet_ECA_B7.zip?dl=0) - - - #### Wide-ResNet-ECA based EfficientNet Architectures: - - [WideResNet-ECA-B1](https://www.dropbox.com/s/pmril11fifyy299/checkpoint_WideResNet_ECA_B1.zip?dl=0) - - [WideResNet-ECA-B2](https://www.dropbox.com/s/j0n9jzcwoh0u8t6/checkpoint_WideResNet_ECA_B2.zip?dl=0) - - [WideResNet-ECA-B3](https://www.dropbox.com/s/w18ay6z05t79o9w/checkpoint_WideResNet_ECA_B3.zip?dl=0) - - [WideResNet-ECA-B4](https://www.dropbox.com/s/tx05ebl3een4ye7/checkpoint_WideResNet_ECA_B4.zip?dl=0) - - [WideResNet-ECA-B5](https://www.dropbox.com/s/ohnn76kp2byyttc/checkpoint_WideResNet_ECA_B5.zip?dl=0) - - [WideResNet-ECA-B6](https://www.dropbox.com/s/gdtudop6dim55c5/checkpoint_WideResNet_ECA_B6.zip?dl=0) - - [WideResNet-ECA-B7](https://www.dropbox.com/s/zh7gnkpvicxulto/checkpoint_WideResNet_ECA_B7.zip?dl=0) - - -## Requirements : - -``` tensorflow==2.4.1 ```, ``` horovod==0.21.0 ``` - -## Usage: +## Available Pretrained Models +All pretrained models can be found [here](https://www.dropbox.com/sh/8bzc1k1e2ejcb7f/AABLwOp_acItoOD7fubevlFKa?dl=0). + +### Standard Architectures +| Architecture | Model Name | Link | F-Score | +| :--- | :--- | :--- | :--- | +| **CNNs** | DenseNet121 | [Download](https://www.dropbox.com/s/idenhh7g4j3vapb/checkpoint_densenet121.zip?dl=0) | 76.8% | +| | DenseNet169 | [Download](https://www.dropbox.com/s/qh6cnryod7uric7/checkpoint_DenseNet169.zip?dl=0) | 75.9% | +| | DenseNet201 | [Download](https://www.dropbox.com/s/w0jdbzdamxfyti7/checkpoint_densenet201.zip?dl=0) | 75.6% | +| | ResNet50 | [Download](https://www.dropbox.com/s/trsojsbjvlpt9jz/checkpoint_resnet50.zip?dl=0) | 76.8% | +| | ResNet101 | [Download](https://www.dropbox.com/s/sn2s5xtqlq9721j/checkpoint_resnet101.zip?dl=0) | 76.3% | +| | ResNet152 | [Download](https://www.dropbox.com/s/ukfsg9s8jxjdb64/checkpoint_resnet152.zip?dl=0) | 75.6% | +| | VGG16 | [Download](https://www.dropbox.com/s/mxx7jm6pleb9ezu/checkpoint_vgg16.zip?dl=0) | 77.3% | +| | VGG19 | [Download](https://www.dropbox.com/s/u86oos6i6vp0981/checkpoint_vgg19.zip?dl=0) | 77.7% | +| **Vision Transformers**| ViT/6 | [Download](https://www.dropbox.com/s/e7ae92x5vitp12e/checkpoint_vit_6.zip?dl=0) | 70.2% | +| | ViT/12 | [Download](https://www.dropbox.com/s/awyw5uz59cu7rh4/checkpoint_vit_12.zip?dl=0) | 76.0% | +| | ViT/20 | [Download](https://www.dropbox.com/s/nkmqvulbb9wbgdw/checkpoint_vit_20.zip?dl=0) | 76.6% | +| | ViT/30 | [Download](https://www.dropbox.com/s/uaayeo2ep7mif3p/checkpoint_vit_30.zip?dl=0) | 76.2% | +| | ViT/40 | [Download](https://www.dropbox.com/s/yyh8dqngk7j52cw/checkpoint_vit_40.zip?dl=0) | 75.3% | +| | ViTM/20 | [Download](https://www.dropbox.com/s/4gtoqfjyar27g69/checkpoint_vit_m20.zip?dl=0) | 77.1% | +| **MLP Mixer** | MLPMixer | [Download](https://www.dropbox.com/s/29lnfpwc0skkirl/checkpoint_MLPMixer.zip?dl=0) | 75.2% | +| | MLPMixer_Tiny | [Download](https://www.dropbox.com/s/lqs69eo512xkkt7/checkpoint_MLPMixer_Tiny.zip?dl=0) | 71.6% | + +### Wide-ResNet Based Architectures +| Model Name | Link | +| :--- | :--- | +| WideResNet | [Download](https://www.dropbox.com/s/g2dah9t9zpk5eda/checkpoint_WideResNet.zip?dl=0) | +| WideResNet-COORD | [Download](https://www.dropbox.com/s/qf9n1i0a48uinez/checkpoint_WideResNet_COORD.zip?dl=0) | +| WideResNet-CBAM | [Download](https://www.dropbox.com/s/ddif8bzfoohx2ap/checkpoint_WideResNet_CBAM.zip?dl=0) | +| WideResNet-CBAM-Ghost | [Download](https://www.dropbox.com/s/ychczklu54ifigs/checkpoint_WideResNet_CBAM_GHOST.zip?dl=0) | +| WideResNet-COORD-Ghost | Link Broken | +| WideResNet-SE | [Download](https://www.dropbox.com/s/9c6wykn88ettxfs/checkpoint_WideResNet_SE.zip?dl=0) | +| WideResNet-SE-Ghost | [Download](https://www.dropbox.com/s/9azmeuhfy1ukb52/checkpoint_WideResNet_SE_GHOST.zip?dl=0) | +| WideResNet-ECA | [Download](https://www.dropbox.com/s/9hqv5xibnx719z0/checkpoint_WideResNet_ECA.zip?dl=0) | +| WideResNet-ECA-GHOST | [Download](https://www.dropbox.com/s/51kzqsrdfhby26v/checkpoint_WideResNet_ECA_GHOST.zip?dl=0) | + +### EfficientNet Family +| Architecture | Model Name | Link | +| :--- | :--- | :--- | +| **Traditional EfficientNet** | EfficientNetB0 | [Download](https://www.dropbox.com/s/bex8c96r29c9c47/checkpoint_EfficientNetB0.zip?dl=0) | +| | EfficientNetB1 | [Download](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) | +| | EfficientNetB2 | [Download](https://www.dropbox.com/s/hcvmq8abhl4qs5r/checkpoint_EfficientNetB1.zip?dl=0) | +| | EfficientNetB3 | [Download](https://www.dropbox.com/s/onmted8umdqv9k9/checkpoint_EfficientNetB3.zip?dl=0) | +| | EfficientNetB4 | [Download](https://www.dropbox.com/s/jtvncero96unys0/checkpoint_EfficientNetB4.zip?dl=0) | +| | EfficientNetB5 | [Download](https://www.dropbox.com/s/yrj930z9y8ecogc/checkpoint_EfficientNetB5.zip?dl=0) | +| | EfficientNetB6 | [Download](https://www.dropbox.com/s/qe3mtzrsv6zb4qx/checkpoint_EfficientNetB6.zip?dl=0) | +| | EfficientNetB7 | [Download](https://www.dropbox.com/s/3x6q4869d2t537f/checkpoint_EfficientNetB7.zip?dl=0) | +| **Augmented EfficientNet**| EfficientNet-CBAM | [Download](https://www.dropbox.com/s/nmd8agsr48fhzox/checkpoint_EfficientNet_CBAM.zip?dl=0) | 76.1% | +| | EfficientNet-CBAM-GHOST | [Download](https://www.dropbox.com/s/aqgn71agf9u44nx/checkpoint_EfficientNet_CBAM_GHOST.zip?dl=0) | +| | EfficientNet-COORD | [Download](https://www.dropbox.com/s/3i1rjwttsxfpqf3/checkpoint_EfficientNet_COORD.zip?dl=0) | +| | EfficientNet-COORD-GHOST| [Download](https://www.dropbox.com/s/5lp6ro9ewfcz0iq/checkpoint_EfficientNet_COORD_GHOST.zip?dl=0) | +| | EfficientNet-SE | [Download](https://www.dropbox.com/s/ubv3ke7qerm2nro/checkpoint_EfficientNet_SE.zip?dl=0) | +| | EfficientNet-SE-GHOST | [Download](https://www.dropbox.com/s/chagaeszrpgeg44/checkpoint_EfficientNet_SE_GHOST.zip?dl=0) | +| | EfficientNet-ECA | [Download](https://www.dropbox.com/s/fnhvuw2uze35tu4/checkpoint_EfficientNet_ECA.zip?dl=0) | +| | EfficientNet-ECA-GHOST | [Download](https://www.dropbox.com/s/3we70krnj06r00a/checkpoint_EfficientNet_ECA_GHOST.zip?dl=0) | +| **ECA EfficientNet** | EfficientNetB1-ECA | [Download](https://www.dropbox.com/s/6p1yqen3at9le2w/checkpoint_EfficientNet_ECA_B1.zip?dl=0) | +| | EfficientNetB2-ECA | [Download](https://www.dropbox.com/s/otfz71k9jpg93d7/checkpoint_EfficientNet_ECA_B2.zip?dl=0) | +| | EfficientNetB3-ECA | [Download](https://www.dropbox.com/s/ncgls9ze7m9v0gn/checkpoint_EfficientNet_ECA_B3.zip?dl=0) | +| | EfficientNetB4-ECA | [Download](https://www.dropbox.com/s/sn5pzw1eryzsid2/checkpoint_EfficientNet_ECA_B4.zip?dl=0) | +| | EfficientNetB5-ECA | [Download](https://www.dropbox.com/s/piwgfovu9bwh91t/checkpoint_EfficientNet_ECA_B5.zip?dl=0) | +| | EfficientNetB6-ECA | [Download](https://www.dropbox.com/s/ryc9hzggwyzmc7k/checkpoint_EfficientNet_ECA_B6.zip?dl=0) | +| | EfficientNetB7-ECA | [Download](https://www.dropbox.com/s/s1a0c2363eh2j9t/checkpoint_EfficientNet_ECA_B7.zip?dl=0) | +| **Wide-ResNet-ECA** | WideResNet-ECA-B1 | [Download](https://www.dropbox.com/s/pmril11fifyy299/checkpoint_WideResNet_ECA_B1.zip?dl=0) | +| | WideResNet-ECA-B2 | [Download](https://www.dropbox.com/s/j0n9jzcwoh0u8t6/checkpoint_WideResNet_ECA_B2.zip?dl=0) | +| | WideResNet-ECA-B3 | [Download](https://www.dropbox.com/s/w18ay6z05t79o9w/checkpoint_WideResNet_ECA_B3.zip?dl=0) | +| | WideResNet-ECA-B4 | [Download](https://www.dropbox.com/s/tx05ebl3een4ye7/checkpoint_WideResNet_ECA_B4.zip?dl=0) | +| | WideResNet-ECA-B5 | [Download](https://www.dropbox.com/s/ohnn76kp2byyttc/checkpoint_WideResNet_ECA_B5.zip?dl=0) | +| | WideResNet-ECA-B6 | [Download](https://www.dropbox.com/s/gdtudop6dim55c5/checkpoint_WideResNet_ECA_B6.zip?dl=0) | +| | WideResNet-ECA-B7 | [Download](https://www.dropbox.com/s/zh7gnkpvicxulto/checkpoint_WideResNet_ECA_B7.zip?dl=0) | + +## Dataset +The dataset used is the BigEarthNet 19 class nomenclature instead of the original 43 class nomenclature. + +## Requirements +`pip install -r .\requirements.txt` + +## Usage To run an experiment modify the [config file](configs/base.json) and execute train.py. Example for MLPMixer with batch size = 100 and learning rate 1e-4: ``` - { "model_name": "MLPMixer", "hparams": {"phi": 1.0, "alpha": 1.0, "beta": 1.0, "gamma": 1.0, "dropout": 0.1}, @@ -133,7 +123,6 @@ abstract = {The availability of the sheer volume of Copernicus Sentinel-2 imager "eval_checkpoint": "/work2/pa20/ipapout/gitSpace/TF1.10.1gpu_Py3/NikosTmp/v2/charmbigearth/bigearthnet-tf2/bestTestResNet50/checkpoint_ResNet50", "augment": true } - ``` To execute in a single-GPU machine: @@ -141,9 +130,7 @@ To execute in a single-GPU machine: python3 train.py --parallel=False ``` -or for multi node training : +or for multi node training: ``` horovodrun --gloo -np $SLURM_NTASKS -H $WORKERS --network-interface ib0 --start-timeout 120 --gloo-timeout-seconds 120 python3 train.py --parallel=True -``` - - +``` \ No newline at end of file