Conference proceeding
M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 37 (NEURIPS 2024), v 37, pp 104694-104723
01 Jan 2024
Abstract
Satellite-based remote sensing has revolutionised the way we address global challenges in a rapidly evolving world. Huge quantities of Earth Observation (EO) data are generated by satellite sensors daily, but processing these large datasets for use in ML pipelines is technically and computationally challenging. Specifically, different types of EO data are often hosted on a variety of platforms, with differing degrees of availability for Python preprocessing tools. In addition, spatial alignment across data sources and data tiling for easier handling can present significant technical hurdles for novice users. While some preprocessed Earth observation datasets exist, their content is often limited to optical or near-optical wavelength data, which is ineffective at night or in adverse weather conditions. Synthetic Aperture Radar (SAR), an active sensing technique based on microwave length radiation, offers a viable alternative. However, the application of machine learning to SAR has been limited due to a lack of ML-ready data and pipelines, particularly for the full diversity of SAR data, including polarimetry, coherence and interferometry. In this work, we introduce M3LEO, a multi-modal, multi-label Earth observation dataset that includes polarimetric, interferometric, and coherence SAR data derived from Sentinel-1, alongside multispectral Sentinel-2 imagery and a suite of auxiliary data describing terrain properties such as land use. M3LEO spans approximately 17M data chips, each measuring 4x4 km, across six diverse geographic regions. The dataset is complemented by a flexible PyTorch Lightning framework, with configuration management using Hydra, to accommodate its use across diverse ML applications in Earth observation. Additionally, we provide tools to process any dataset available on popular platforms such as Google Earth Engine for seamless integration with our framework. We show that the distribution shift in self-supervised embeddings is substantial across geographic regions, even when controlling for terrain properties. Data is available at huggingface.co/M3LEO, and code at github.com/spaceml-org/M3LEO.
Metrics
13 Record Views
Details
- Title
- M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data
- Creators
- Matt Allen - University of CambridgeFrancisco DorrJoseph Gallego Mejia - Drexel UniversityLaura Martinez-Ferrer - Universitat de ValènciaAnna Jungbluth - European Space AgencyFreddie Kalaitzis - University of OxfordRaul Ramos-Pollan - Universidad de Antioquia
- Publication Details
- ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 37 (NEURIPS 2024), v 37, pp 104694-104723
- Conference
- Advances in Neural Information Processing Systems 37, 37 (Vancouver, Canada, 10 Dec 2024–15 Dec 2024)
- Series
- Advances in Neural Information Processing Systems
- Publisher
- Neural Information Processing Systems (Nips)
- Number of pages
- 30
- Grant note
- European Space Agency (ESA); European Space Agency Google Cloud and Nvidia, developing open science for all Humankind EP/S022961/1 / UKRI Centre for Doctoral Training in Application of Artificial Intelligence 855187 / European Research Council (ERC) Synergy Grant "Understanding and Modelling the Earth System with Machine Learning (USMILE)" under the Horizon 2020 research and innovation programme
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science
- Web of Science ID
- WOS:001633227600143
- Scopus ID
- 2-s2.0-105000488282
- Other Identifier
- 991022116574004721