Self-Supervised Masked DEM Encoding

Written By Priyam Mazumdar

Priyam Mazumdar [1,2,3]     Dr. Aiman Soliman [2,3,4]

[1] Department of Electrical Engineering, University of Illinois Urbana-Champaign

[2] National Center for Supercomputing Applications, University of Illinois Urbana-Champaign

[3] Center for Artificial Intelligence Innovation (CAII), University of Illinois Urbana-Champaign

[4] Department of Urban and Regional Planning, University of Illinois Urbana-Champaign

Abstract

The lack of labeled data is one of the main bottlenecks for training machine and deep learning (DL) models. To combat this we utilize the self-supervised Masked AutoEncoder (MAE) using manually labeled benchmarks for the task of segmenting building footprints from digital elevation models (DEM). The MAE architecture utilizes the standard autoencoder framework through a Vision Transformer based encoder/decoder, and segmentation is generated from an UperNet head. This MAE backbone is first pre-trained on all available (unlabeled) DEM images and then fine-tuned to limited segmentation masks. Our initial results yield nearly an 82% validation IOU when trained on only 450 hand-labeled images for building prediction. Although performance continues to drop, we also retain an acceptable 69% validation IOU when trained on 50 Images. Training sizes under 50 continue to return general placement of the buildings in an image but not accurately match their shapes. These results demonstrate the possibility to use limited datasets for more niche tasks when training large DL models and overcome the scarcity of quality labels.

Next Steps

There is still a lot of exciting work to be done though! We are currently in the process of performing PreTraining on the MAE model with terabytes of data sampled from the Arctic Region. Once this is done we have a couple of plans for different downstream tasks

  • Land use classification (Finetuned with CLS Tokens)

  • Object Detection (Detectron2 with MAE Backbone)

    • Building Type Detection

    • Environmental Feature Detection

  • Segmentation (MAE + UperNet)

    • Road segmentation

    • Spatial Environmental Analysis

    • Anomoly Detection

Acknowledgements

I want to thank the National Center for Super Computing Applications along with the Permafrost Project for

Priyam Mazumdar
Next

DARPA Map Feature Extraction Challenge