Detecting floating objects using deep learning and Sentinel-2 imagery
Contents
Detecting floating objects using deep learning and Sentinel-2 imagery¶
Ocean Modelling Standard Python
Context¶
Purpose¶
Detect floating targets that might contain plastic, algea, sargassum, wood, etc. in coastal areas using deep learning methods and Sentinel-2 data.
Modelling approach¶
In this notebook we show the potential of deep learning methods to detect and highlight floating debris of various natures in coastal areas. First, the dataset can be downloaded from Zenodo using this notebook and it contains: a Sentinel-2 image over a coastal area of Rio de Janeiro in Brazil, a validation example from the Plastic Litter Project (PLP), and two use-case Sentinel-2 images of scenes containing various floating targets. We then apply pre-trained weights in order to identify the potential existence of floating targets. We visualise the results in an interactive way and we also show that the model works on validation data and on examples containing different types of floating objects. We finally export the predictions in Geotiff formats to the output folders. Our experiments were implemented using Pytorch. Further details on the labeled dataset and the code can be found here: https://github.com/ESA-PhiLab/floatingobjects.
Highlights¶
We demonstrate the use of deep neural networks for the detection of floating objects on Sentinel-2 data.
Once the user downloads the dataset from Zenodo using this notebook, the predictions can be run and the results will be visualised.
Several use cases are included in order to show the variety of the detected objects.
The user can visualise the RGB image, the NDVI and FDI indices along with the predictions and classifications.
The predictions will be available locally in the user’s folder.
Contributions¶
Notebook¶
Jamila Mifdal (author), European Space Agency Φ-lab, @jmifdal
Raquel Carmo (author), European Space Agency Φ-lab, @raquelcarmo
Alejandro Coca-Castro (reviewer), The Alan Turing Institute, @acocac
Modelling codebase¶
Jamila Mifdal (author), European Space Agency Φ-lab, @jmifdal
Raquel Carmo (author), European Space Agency Φ-lab, @raquelcarmo
Marc Rußwurm (author), EPFL-ECEO, @marccoru
Modelling publications¶
Raquel Carmo, Jamila Mifdal, and Marc Rußwurm. Detecting macro floating objects on coastal water bodies using sentinel-2 data. In OCEANS 2021. OCEANS 2021, 2021. URL: https://210507-004.oceansvirtual.com/view/content/skdwP611e3583eba2b/ecf65c2aaf278557ad05c213247d67a54196c9376a0aed8f1875681f182daeed.
Jamila Mifdal, Nicolas Longepe, and Marc Rußwurm. Towards detecting floating objects on a global scale with learned spatial features using sentinel 2. In ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, volume 5, 285–293. Copernicus GmbH, 6 2021. doi:10.5194/isprs-annals-V-3-2021-285-2021.
Modelling funding¶
This project is supported by Φ-lab, European Space Agency.
Install and load libraries¶
!pip -q install gdown
WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
import os
import numpy as np
# intake library and plugin
import intake
from intake_zenodo_fetcher import download_zenodo_files_for_entry
# xarray
import xarray as xr
# machine learning libraries
import torch
# image libraries
from PIL import Image
# visualisation
import rasterio as rio
import matplotlib, matplotlib.cm
import matplotlib.pyplot as plt
from skimage.exposure import equalize_hist
import holoviews as hv
import hvplot.xarray
import warnings
warnings.filterwarnings(action='ignore')
hv.extension('bokeh', width=100)