MODIS MOD021KM and FIRMS

Wildfires Modelling Standard Python

Context

Purpose

Explore MODIS satellite imagery and wildfire data that is open and free for scientific use.

Sensor description

The MOD021KM product contains calibrated and geolocated at-aperture radiances for 36 discrete bands located in the 0.4 to 14.4 micron region of the electromagnetic spectrum.

Highlights

• Use satpy to load, visualise, and regrid MODIS level 1B data.

• Fetch a fire database containing some 497364 fires from 2020.

• Visualisation of fire pixels from the database.

• Visualisation of the fire pixels alongside bands from the MODIS satellite data.

Contributions

Notebook

• Samuel Jackson (author), Science & Technology Facilities Council, @samueljackson92

• Alejandro Coca-Castro (reviewer), The Alan Turing Institute, @acocac

Dataset originator/creator

MOD021KM

• MODIS Characterization Support Team (MCST)

• MODIS Adaptive Processing System (MODAPS)

Firms

• University of Maryland

Dataset authors

MOD021KM

• MODIS Science Data Support Team (SDST)

Firms

• NASA’s Applied Sciences Program

Dataset documentation

• Louis Giglio, Wilfrid Schroeder, Joanne V. Hall, and Christopher O. Justice. MODIS Collection 6 Active Fire Product User’s Guide Revision B. Technical Report, NASA, 2018. URL: https://modis-fire.umd.edu/files/MODIS_C6_Fire_User_Guide_B.pdf.

Note

The author acknowledges MODIS Science Team and the use of data and/or imagery from NASA’s Fire Information for Resource Management System (FIRMS) (https://earthdata.nasa.gov/firms), part of NASA’s Earth Observing System Data and Information System (EOSDIS).

Install and load libraries

Show code cell source Hide code cell source

!pip -q install geopandas
!pip -q install geoviews

WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Show code cell source Hide code cell source

import pandas as pd
import numpy as np
import geopandas
import intake
import fsspec, aiohttp
import hvplot.xarray
import hvplot.pandas
import holoviews as hv
import panel as pn
import satpy
import xarray as xr
import tempfile
from pathlib import Path
from scipy.spatial import cKDTree
from satpy.writers import get_enhanced_image
from getpass import getpass
from pathlib import Path
from pyresample import geometry
import datetime
import urllib.request
import os.path
from dotenv import load_dotenv
import warnings
warnings.filterwarnings(action='ignore')

Set project structure

notebook_folder = './notebook'
if not os.path.exists(notebook_folder):
    os.makedirs(notebook_folder)

Fetch Data

Note

To download data from the NASA’s Earth Data site you must have a valid Earth Data account. Please register with Earth Data is you do not already have an account and then provide your username and password when prompted in the cell below.

Important

In order to download MODIS level1 data you must have authorized access to LAADS Web on your account. To do this navigate to your earthdata profile page

• Click “Applications”

• Click “Authorized Apps”

• Click “Autorize More Applications”

• Search for “LAADS Web”

• Click “Authorize”

Now you should successfully be able to download MODIS data.

fname = 'MOD021KM.A2020245.0840.061.2020245193036.hdf'

if not os.path.isfile(os.path.join(notebook_folder, fname)) or os.path.getsize(os.path.join(notebook_folder, fname)) == 0:
    load_dotenv()
    username = os.environ.get("NASA_EARTHDATA_USER") #replace for your EarthData username if run local or in Binder
    password = os.environ.get("NASA_EARTHDATA_PWD")  #replace for your EarthData password if run local or in Binder

    fsspec.config.conf['https'] = dict(client_kwargs={'auth': aiohttp.BasicAuth(username, password)})

    url = f'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD021KM/2020/245/{fname}'
    filename = url.split('/')[-1]
    with fsspec.open(url) as f:
        with Path(os.path.join(notebook_folder, filename)).open('wb') as handle:
            data = f.read()
            try:
                data.decode('utf-8')
                raise RuntimeError('Could not download MODIS data! Have you authorized LAADS Web in your Eathdata account above?')
            except UnicodeDecodeError:
                handle.write(data)

Download the database of MODIS wildfires from a publically accessible location.

filepath = 'https://firms2.modaps.eosdis.nasa.gov/data/country/zips'
filename = 'modis_2020_all_countries.zip'

if not os.path.isfile(filename) or os.path.getsize(filename) == 0:
    urllib.request.urlretrieve(os.path.join(filepath, filename), os.path.join(notebook_folder, filename))
    !unzip -o ./notebook/modis_2020_all_countries.zip -d ./notebook

Archive:  ./notebook/modis_2020_all_countries.zip
   creating: ./notebook/modis/2020/
  inflating: ./notebook/modis/2020/modis_2020_Afghanistan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Albania.csv  
  inflating: ./notebook/modis/2020/modis_2020_Algeria.csv  
  inflating: ./notebook/modis/2020/modis_2020_Andorra.csv  
  inflating: ./notebook/modis/2020/modis_2020_Angola.csv  
  inflating: ./notebook/modis/2020/modis_2020_Antigua_and_Barbuda.csv  
  inflating: ./notebook/modis/2020/modis_2020_Argentina.csv  
  inflating: ./notebook/modis/2020/modis_2020_Armenia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Australia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Austria.csv  
  inflating: ./notebook/modis/2020/modis_2020_Azerbaijan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bahamas.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bahrain.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bangladesh.csv  
  inflating: ./notebook/modis/2020/modis_2020_Barbados.csv  
  inflating: ./notebook/modis/2020/modis_2020_Belarus.csv  
  inflating: ./notebook/modis/2020/modis_2020_Belgium.csv  
  inflating: ./notebook/modis/2020/modis_2020_Belize.csv  
  inflating: ./notebook/modis/2020/modis_2020_Benin.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bermuda.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bhutan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bolivia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bosnia_and_Herzegovina.csv  
  inflating: ./notebook/modis/2020/modis_2020_Botswana.csv  
  inflating: ./notebook/modis/2020/modis_2020_Brazil.csv  
  inflating: ./notebook/modis/2020/modis_2020_British_Virgin_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Brunei_Darussalam.csv  
  inflating: ./notebook/modis/2020/modis_2020_Bulgaria.csv  
  inflating: ./notebook/modis/2020/modis_2020_Burkina_Faso.csv  
  inflating: ./notebook/modis/2020/modis_2020_Burundi.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cambodia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cameroon.csv  
  inflating: ./notebook/modis/2020/modis_2020_Canada.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cape_Verde.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cayman_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Central_African_Republic.csv  
  inflating: ./notebook/modis/2020/modis_2020_Chad.csv  
  inflating: ./notebook/modis/2020/modis_2020_Chile.csv  
  inflating: ./notebook/modis/2020/modis_2020_China.csv  
  inflating: ./notebook/modis/2020/modis_2020_Colombia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Comoros.csv  
  inflating: ./notebook/modis/2020/modis_2020_Costa_Rica.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cote_d_Ivoire.csv  
  inflating: ./notebook/modis/2020/modis_2020_Croatia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cuba.csv  
  inflating: ./notebook/modis/2020/modis_2020_Curacao.csv  
  inflating: ./notebook/modis/2020/modis_2020_Cyprus.csv  
  inflating: ./notebook/modis/2020/modis_2020_Czech_Republic.csv  
  inflating: ./notebook/modis/2020/modis_2020_Dem_Rep_Korea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Democratic_Republic_of_the_Congo.csv  
  inflating: ./notebook/modis/2020/modis_2020_Denmark.csv  
  inflating: ./notebook/modis/2020/modis_2020_Djibouti.csv  
  inflating: ./notebook/modis/2020/modis_2020_Dominica.csv  
  inflating: ./notebook/modis/2020/modis_2020_Dominican_Republic.csv  
  inflating: ./notebook/modis/2020/modis_2020_Ecuador.csv  
  inflating: ./notebook/modis/2020/modis_2020_Egypt.csv  
  inflating: ./notebook/modis/2020/modis_2020_El_Salvador.csv  
  inflating: ./notebook/modis/2020/modis_2020_Equatorial_Guinea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Eritrea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Estonia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Ethiopia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Falkland_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Fiji.csv  
  inflating: ./notebook/modis/2020/modis_2020_Finland.csv  
  inflating: ./notebook/modis/2020/modis_2020_France.csv  
  inflating: ./notebook/modis/2020/modis_2020_French_Guiana.csv  
  inflating: ./notebook/modis/2020/modis_2020_French_Polynesia.csv  
  inflating: ./notebook/modis/2020/modis_2020_French_Southern_and_Antarctic_Lands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Gabon.csv  
  inflating: ./notebook/modis/2020/modis_2020_Georgia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Germany.csv  
  inflating: ./notebook/modis/2020/modis_2020_Ghana.csv  
  inflating: ./notebook/modis/2020/modis_2020_Greece.csv  
  inflating: ./notebook/modis/2020/modis_2020_Greenland.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guadeloupe.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guam.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guatemala.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guinea-Bissau.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guinea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Guyana.csv  
  inflating: ./notebook/modis/2020/modis_2020_Haiti.csv  
  inflating: ./notebook/modis/2020/modis_2020_Heard_I_and_McDonald_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Honduras.csv  
  inflating: ./notebook/modis/2020/modis_2020_Hong_Kong.csv  
  inflating: ./notebook/modis/2020/modis_2020_Hungary.csv  
  inflating: ./notebook/modis/2020/modis_2020_India.csv  
  inflating: ./notebook/modis/2020/modis_2020_Indonesia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Iran.csv  
  inflating: ./notebook/modis/2020/modis_2020_Iraq.csv  
  inflating: ./notebook/modis/2020/modis_2020_Ireland.csv  
  inflating: ./notebook/modis/2020/modis_2020_Israel.csv  
  inflating: ./notebook/modis/2020/modis_2020_Italy.csv  
  inflating: ./notebook/modis/2020/modis_2020_Jamaica.csv  
  inflating: ./notebook/modis/2020/modis_2020_Japan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Jordan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kazakhstan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kenya.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kiribati.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kosovo.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kuwait.csv  
  inflating: ./notebook/modis/2020/modis_2020_Kyrgyzstan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Lao_PDR.csv  
  inflating: ./notebook/modis/2020/modis_2020_Latvia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Lebanon.csv  
  inflating: ./notebook/modis/2020/modis_2020_Lesotho.csv  
  inflating: ./notebook/modis/2020/modis_2020_Liberia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Libya.csv  
  inflating: ./notebook/modis/2020/modis_2020_Lithuania.csv  
  inflating: ./notebook/modis/2020/modis_2020_Luxembourg.csv  
  inflating: ./notebook/modis/2020/modis_2020_Macedonia_Former_Yugoslav_Republic_of.csv  
  inflating: ./notebook/modis/2020/modis_2020_Madagascar.csv  
  inflating: ./notebook/modis/2020/modis_2020_Malawi.csv  
  inflating: ./notebook/modis/2020/modis_2020_Malaysia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Maldives.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mali.csv  
  inflating: ./notebook/modis/2020/modis_2020_Malta.csv  
  inflating: ./notebook/modis/2020/modis_2020_Martinique.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mauritania.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mauritius.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mexico.csv  
  inflating: ./notebook/modis/2020/modis_2020_Moldova.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mongolia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Montenegro.csv  
  inflating: ./notebook/modis/2020/modis_2020_Montserrat.csv  
  inflating: ./notebook/modis/2020/modis_2020_Morocco.csv  
  inflating: ./notebook/modis/2020/modis_2020_Mozambique.csv  
  inflating: ./notebook/modis/2020/modis_2020_Myanmar.csv  
  inflating: ./notebook/modis/2020/modis_2020_Namibia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Nepal.csv  
  inflating: ./notebook/modis/2020/modis_2020_Netherlands.csv  
  inflating: ./notebook/modis/2020/modis_2020_New_Caledonia.csv  
  inflating: ./notebook/modis/2020/modis_2020_New_Zealand.csv  
  inflating: ./notebook/modis/2020/modis_2020_Nicaragua.csv  
  inflating: ./notebook/modis/2020/modis_2020_Niger.csv  
  inflating: ./notebook/modis/2020/modis_2020_Nigeria.csv  
  inflating: ./notebook/modis/2020/modis_2020_Northern_Mariana_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Norway.csv  
  inflating: ./notebook/modis/2020/modis_2020_Oman.csv  
  inflating: ./notebook/modis/2020/modis_2020_Pakistan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Palau.csv  
  inflating: ./notebook/modis/2020/modis_2020_Palestine.csv  
  inflating: ./notebook/modis/2020/modis_2020_Panama.csv  
  inflating: ./notebook/modis/2020/modis_2020_Papua_New_Guinea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Paraguay.csv  
  inflating: ./notebook/modis/2020/modis_2020_Peru.csv  
  inflating: ./notebook/modis/2020/modis_2020_Philippines.csv  
  inflating: ./notebook/modis/2020/modis_2020_Poland.csv  
  inflating: ./notebook/modis/2020/modis_2020_Portugal.csv  
  inflating: ./notebook/modis/2020/modis_2020_Puerto_Rico.csv  
  inflating: ./notebook/modis/2020/modis_2020_Qatar.csv  
  inflating: ./notebook/modis/2020/modis_2020_Republic_of_Congo.csv  
  inflating: ./notebook/modis/2020/modis_2020_Republic_of_Korea.csv  
  inflating: ./notebook/modis/2020/modis_2020_Reunion.csv  
  inflating: ./notebook/modis/2020/modis_2020_Romania.csv  
  inflating: ./notebook/modis/2020/modis_2020_Russian_Federation.csv  
  inflating: ./notebook/modis/2020/modis_2020_Rwanda.csv  
  inflating: ./notebook/modis/2020/modis_2020_Saint_Helena.csv  
  inflating: ./notebook/modis/2020/modis_2020_Saint_Kitts_and_Nevis.csv  
  inflating: ./notebook/modis/2020/modis_2020_Saint_Lucia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Saint_Vincent_and_the_Grenadines.csv  
  inflating: ./notebook/modis/2020/modis_2020_Samoa.csv  
  inflating: ./notebook/modis/2020/modis_2020_Sao_Tome_and_Principe.csv  
  inflating: ./notebook/modis/2020/modis_2020_Saudi_Arabia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Senegal.csv  
  inflating: ./notebook/modis/2020/modis_2020_Serbia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Seychelles.csv  
  inflating: ./notebook/modis/2020/modis_2020_Sierra_Leone.csv  
  inflating: ./notebook/modis/2020/modis_2020_Singapore.csv  
  inflating: ./notebook/modis/2020/modis_2020_Slovakia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Slovenia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Solomon_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Somalia.csv  
  inflating: ./notebook/modis/2020/AttributesReadme.txt  
  inflating: ./notebook/modis/2020/modis_2020_South_Africa.csv  
  inflating: ./notebook/modis/2020/modis_2020_South_Sudan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Spain.csv  
  inflating: ./notebook/modis/2020/modis_2020_Sri_Lanka.csv  
  inflating: ./notebook/modis/2020/modis_2020_Sudan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Suriname.csv  
  inflating: ./notebook/modis/2020/modis_2020_Svalbard_and_Jan_Mayen.csv  
  inflating: ./notebook/modis/2020/modis_2020_Swaziland.csv  
  inflating: ./notebook/modis/2020/modis_2020_Sweden.csv  
  inflating: ./notebook/modis/2020/modis_2020_Switzerland.csv  
  inflating: ./notebook/modis/2020/modis_2020_Syria.csv  
  inflating: ./notebook/modis/2020/modis_2020_Taiwan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Tajikistan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Tanzania.csv  
  inflating: ./notebook/modis/2020/modis_2020_Thailand.csv  
  inflating: ./notebook/modis/2020/modis_2020_The_Gambia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Timor-Leste.csv  
  inflating: ./notebook/modis/2020/modis_2020_Togo.csv  
  inflating: ./notebook/modis/2020/modis_2020_Tonga.csv  
  inflating: ./notebook/modis/2020/modis_2020_Trinidad_and_Tobago.csv  
  inflating: ./notebook/modis/2020/modis_2020_Tunisia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Turkey.csv  
  inflating: ./notebook/modis/2020/modis_2020_Turkmenistan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Turks_and_Caicos_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Uganda.csv  
  inflating: ./notebook/modis/2020/modis_2020_Ukraine.csv  
  inflating: ./notebook/modis/2020/modis_2020_United_Arab_Emirates.csv  
  inflating: ./notebook/modis/2020/modis_2020_United_Kingdom.csv  
  inflating: ./notebook/modis/2020/modis_2020_United_States.csv  
  inflating: ./notebook/modis/2020/modis_2020_United_States_Minor_Outlying_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_United_States_Virgin_Islands.csv  
  inflating: ./notebook/modis/2020/modis_2020_Uruguay.csv  
  inflating: ./notebook/modis/2020/modis_2020_Uzbekistan.csv  
  inflating: ./notebook/modis/2020/modis_2020_Vanuatu.csv  
  inflating: ./notebook/modis/2020/modis_2020_Venezuela.csv  
  inflating: ./notebook/modis/2020/modis_2020_Vietnam.csv  
  inflating: ./notebook/modis/2020/modis_2020_Yemen.csv  
  inflating: ./notebook/modis/2020/modis_2020_Zambia.csv  
  inflating: ./notebook/modis/2020/modis_2020_Zimbabwe.csv  

Load an intake catalog for the downloaded data

# set catalogue location
catalog_file = os.path.join(notebook_folder, 'catalog.yaml')

with open(catalog_file, 'w') as f:
    f.write('''
sources:
    modis_l1b:
      args:
        urlpath: 'notebook/MOD021KM.A2020245.0840.061.2020245193036.hdf'
        reader: 'modis_l1b'
      description: "MODIS Level 1B Products"
      driver: SatpySource
      metadata: {}
    modis_fires:
      args:
        urlpath: 'notebook/modis/2020/modis_*.csv'
      description: "MODIS Level 2 Fires"
      driver: csv
      metadata: {}
''')

from intake.source.base import PatternMixin
from intake.source.base import DataSource, Schema

import glob

class SatpySource(DataSource, PatternMixin):
    """Intake driver for data supported by satpy.Scene"""
    
    container = 'python'
    name = 'satpy'
    version = '0.0.1'
    partition_access = False

    def __init__(self, urlpath, reader=None, metadata=None, path_as_pattern=True):
        """
        Parameters
        ----------
        path: str, location of model pkl file
        Either the absolute or relative path to the file
        opened. Some examples:
          - ``{{ CATALOG_DIR }}/file.nat``
          - ``{{ CATALOG_DIR }}/*.nc``
        reader: str, optional
        Name of the satpy reader to use when loading data (ie. ``modis_l1b``)
        metadata: dict, optional
        Additional metadata to pass to the intake catalog
        path_as_pattern: bool or str, optional
        Whether to treat the path as a pattern (ie. ``data_{field}.nc``)
        and create new coodinates in the output corresponding to pattern
        fields. If str, is treated as pattern to match on. Default is True.
        """

        self.path_as_pattern = path_as_pattern
        self.urlpath = urlpath
        self._reader = reader
        super(SatpySource, self).__init__(metadata=metadata)

    def _load(self):
        files = self.urlpath
        files = list(glob.glob(files))
        return satpy.Scene(files, reader=self._reader)
    
    def _get_schema(self):
        self._schema = Schema(
            npartitions=1,
            extra_metadata={}
        )
        return self._schema

    def read(self):
        self._load_metadata()
        return self._load()

intake.register_driver('SatpySource', SatpySource)
cat = intake.open_catalog(catalog_file)

Load MODIS Satellite Data

Here we use satpy to load the MODIS level 1b data into memory. As well as loading the image data, satpy provides a easy way to regrid the data to different coordinate systems.

scn = cat['modis_l1b'].read()
scn

<satpy.scene.Scene at 0x798e8f378e20>

Show code cell source Hide code cell source

scn.load(['true_color', '20'], resolution=1000)
plot_modis_raw = scn.show('true_color')
plot_modis_raw

100%|██████████| 52/52 [00:00<00:00, 954.03kB/s]
No rsr file /home/jovyan/.local/share/pyspectral/rsr_modis_EOS-Terra.h5 on disk
100%|██████████| 5/5 [00:00<00:00, 779.29kB/s]

Resample to a different projection

In the plot above you can see that the raw MODIS data has distortion towards edge of the image. By regridding the data we can avoid some of this distortion.

area_id = "Southern Africa"
description = "Southern Africa in Mercator-Projection"
proj_id = "Southern Africa"
proj_dict = {"proj": "eqc"}

width = 1000    # width of the result domain in pixels
height = 1000   # height of the result domain in pixels

llx =  23E5   # projection x coordinate of lower left corner of lower left pixel
lly =  -22.9E5   # projection y coordinate of lower left corner of lower left pixel
urx =  48E5   # projection x coordinate of upper right corner of upper right pixel
ury =  -1.9E5   # projection y coordinate of upper right corner of upper right pixel

area_extent = (llx,lly,urx,ury)
from pyresample import create_area_def
resolution=1000
center =(0,0)
area_def = create_area_def(area_id, proj_dict, center=center, resolution=resolution)

modis_scn = scn.resample(area_def, radius_of_influence=10000)

Show code cell source Hide code cell source

plot_modis_scn = modis_scn.show('true_color')
plot_modis_scn

Load MODIS Fire Database

Now we’re going to load the modis fire database from CSV files. This contains the longitude, latitude, and time of where fires have been detected to occur. It also includes an estimate of the Fire Radiative Power (FRP), a measure of the intensity of the fire, for each fire detected.

Show code cell source Hide code cell source

fires = cat['modis_fires'].read()

time = fires.acq_date.astype(str) + ' ' + fires.acq_time.astype(str).str.zfill(4)
fires['timestamp'] = pd.to_datetime(time, format='%Y-%m-%d %H%M')
fires = fires.loc[fires.satellite == 'Terra']
fires = geopandas.GeoDataFrame(
    fires, geometry=geopandas.points_from_xy(fires.longitude, fires.latitude))

# We're only interested in data from Southern Africa for now
llx =  0    # projection x coordinate of lower left corner of lower left pixel
lly =  -30  # projection y coordinate of lower left corner of lower left pixel
urx =  55   # projection x coordinate of upper right corner of upper right pixel
ury =  10   # projection y coordinate of upper right corner of upper right pixel

fires = fires.cx[llx:urx, lly:ury]
fires.set_index('timestamp', drop=False, inplace=True)
fires = fires.sort_index()
fires = fires.loc['2020-09-01 00:00:00':'2020-10-01 00:00:00']
fires

	latitude	longitude	brightness	scan	track	acq_date	acq_time	satellite	instrument	confidence	version	bright_t31	frp	daynight	type	timestamp	geometry
timestamp
2020-09-01 07:02:00	-15.8201	46.6282	320.8	1.2	1.1	2020-09-01	702	Terra	MODIS	59	6.03	306.2	8.7	D	0	2020-09-01 07:02:00	POINT (46.62820 -15.82010)
2020-09-01 07:02:00	-15.8183	46.6171	323.7	1.2	1.1	2020-09-01	702	Terra	MODIS	67	6.03	307.9	12.7	D	0	2020-09-01 07:02:00	POINT (46.61710 -15.81830)
2020-09-01 07:02:00	-15.7672	47.2667	324.8	1.1	1.1	2020-09-01	702	Terra	MODIS	72	6.03	307.6	9.9	D	0	2020-09-01 07:02:00	POINT (47.26670 -15.76720)
2020-09-01 07:02:00	-15.7055	47.7069	328.2	1.1	1.0	2020-09-01	702	Terra	MODIS	75	6.03	306.8	15.4	D	0	2020-09-01 07:02:00	POINT (47.70690 -15.70550)
2020-09-01 07:02:00	-15.0942	47.2146	320.4	1.2	1.1	2020-09-01	702	Terra	MODIS	65	6.03	302.7	9.6	D	0	2020-09-01 07:02:00	POINT (47.21460 -15.09420)
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2020-09-30 22:05:00	-4.5185	12.0523	306.7	1.8	1.3	2020-09-30	2205	Terra	MODIS	69	6.03	288.2	17.6	N	2	2020-09-30 22:05:00	POINT (12.05230 -4.51850)
2020-09-30 22:06:00	-2.7462	10.7768	302.2	1.5	1.2	2020-09-30	2206	Terra	MODIS	47	6.03	287.3	8.6	N	0	2020-09-30 22:06:00	POINT (10.77680 -2.74620)
2020-09-30 22:06:00	-0.8865	8.8320	302.2	1.2	1.1	2020-09-30	2206	Terra	MODIS	48	6.03	289.0	4.9	N	0	2020-09-30 22:06:00	POINT (8.83200 -0.88650)
2020-09-30 22:06:00	-2.8421	10.8256	301.0	1.5	1.2	2020-09-30	2206	Terra	MODIS	37	6.03	287.2	7.8	N	0	2020-09-30 22:06:00	POINT (10.82560 -2.84210)
2020-09-30 22:08:00	4.4222	7.1571	302.5	1.1	1.1	2020-09-30	2208	Terra	MODIS	49	6.03	291.9	4.5	N	2	2020-09-30 22:08:00	POINT (7.15710 4.42220)

65459 rows × 17 columns

Daily Fire Radiative Power

Here we plot the Fire Radiative Power (FRP) for each day of the month of September. FRP is a measure of the intensity of a fire in units of MegaWatts.

Show code cell source Hide code cell source

from bokeh.models.formatters import DatetimeTickFormatter

def plot_timeseries(data, y_variable):
    """Timeseries plot showing the mean fire radiative power at each day as well as the 5% and 95%"""

    def percentile(n):
        def percentile_(x):
            return np.percentile(x, n)
        percentile_.__name__ = 'percentile_%s' % n
        return percentile_

    formatter = DatetimeTickFormatter(months='%b')

    bounds = data.groupby([data.index.day_of_year])[y_variable].agg([percentile(5), percentile(95)])
    avg = data.groupby([data.index.day_of_year])[y_variable].mean()

    bounds.index = data.groupby([data.index.day_of_year])['timestamp'].first()
    avg.index = bounds.index

    tseries = hv.Overlay([
        (bounds.hvplot.area('timestamp', 'percentile_5', 'percentile_95',
                      alpha=0.2, color='red', xformatter=formatter)),
            avg.hvplot.line(x='timestamp', label=f'Average Daily FRP', color='red', xformatter=formatter)])

    return tseries.options(width=800, height=400, xlabel='Day',ylabel=f'FRP (MW)',legend_position='top_left')

plot_FIRMS_ts = plot_timeseries(fires, 'frp')
plot_FIRMS_ts

Geographical distribution of Fires

Here we plot the geographical distribution of fires detected by MODIS over Southern Africa.

Show code cell source Hide code cell source

plot_FIRMS_geo = fires.hvplot.points('longitude', 'latitude', groupby='index.day_of_year', c='frp', geo=True, alpha=0.2,
                    tiles='ESRI', xlim=(llx, urx), ylim=(lly, ury), cmap='autumn', logz=True,
                    dynamic=False)
plot_FIRMS_geo

WARNING:param.GeoPointPlot03910: Log color mapper lower bound <= 0 and will not render correctly. Ensure you set a positive lower bound on the color dimension or using the `clim` option.
WARNING:param.GeoPointPlot03910: Log color mapper lower bound <= 0 and will not render correctly. Ensure you set a positive lower bound on the color dimension or using the `clim` option.

Visualising Fire Pixels with Satellite Imagery

Visualise a color image of the MODIS region using hvplot.

Show code cell source Hide code cell source

modis_dataset = modis_scn.to_xarray_dataset()

img = get_enhanced_image(modis_scn['true_color'].squeeze())
img = img.data
img = img.clip(0, 1)
img = (img * 255).astype(np.uint8)

modis_dataset['true_color'] = img

grid = modis_scn.min_area().get_lonlats()
lons, lats = grid[0][0], grid[1][:, 0]
modis_dataset = modis_dataset.assign_coords(dict(x=lons, y=lats))
modis_dataset = modis_dataset.rename(dict(x='longitude', y='latitude'))

plot_MODIS_rgb = modis_dataset['true_color'].hvplot.rgb(x='longitude', y='latitude', bands='bands', rasterize=True)
plot_MODIS_rgb

Get the fire pixels that are visible in the the MODIS scene by filtering the time, longitude & latitude. We can also compute which row and column the fire pixel is in for this projection.

Show code cell source Hide code cell source

grid = modis_scn.min_area().get_lonlats()

# Mask any fire pixels not in this scene
time_mask = np.logical_and(fires.timestamp >= scn.start_time,
                           fires.timestamp <= scn.end_time)
fire_points = fires.loc[time_mask]
fire_points = fire_points.cx[grid[0].min():grid[0].max(), grid[1].min():grid[1].max()]

# extract the x and y coordinates as flat arrays
x = np.ravel(grid[0])
y = np.ravel(grid[1])
points = np.dstack([x, y]).squeeze()

# Find locations of fire pixels within satellite image
tree = cKDTree(points)
distance, idx = tree.query(fire_points[['longitude', 'latitude']], k=1)
index = np.unravel_index(idx, grid[0].shape)
index = np.vstack(index).T
index

# fire_points[['y', 'x']] = index
fire_points = fire_points[img.values[0, index[:, 0], index[:, 1]] != 0]
fire_points

	latitude	longitude	brightness	scan	track	acq_date	acq_time	satellite	instrument	confidence	version	bright_t31	frp	daynight	type	timestamp	geometry
timestamp
2020-09-01 08:40:00	-12.0362	30.9757	321.5	1.8	1.3	2020-09-01	840	Terra	MODIS	67	6.03	305.2	20.0	D	0	2020-09-01 08:40:00	POINT (30.97570 -12.03620)
2020-09-01 08:40:00	-11.6813	29.0762	328.7	1.3	1.1	2020-09-01	840	Terra	MODIS	79	6.03	307.0	20.8	D	0	2020-09-01 08:40:00	POINT (29.07620 -11.68130)
2020-09-01 08:40:00	-11.9208	31.1634	323.1	1.8	1.3	2020-09-01	840	Terra	MODIS	65	6.03	306.3	24.3	D	0	2020-09-01 08:40:00	POINT (31.16340 -11.92080)
2020-09-01 08:40:00	-12.0026	30.6289	326.0	1.7	1.3	2020-09-01	840	Terra	MODIS	75	6.03	304.3	31.9	D	0	2020-09-01 08:40:00	POINT (30.62890 -12.00260)
2020-09-01 08:40:00	-11.7261	29.1128	325.1	1.3	1.1	2020-09-01	840	Terra	MODIS	71	6.03	304.9	16.8	D	0	2020-09-01 08:40:00	POINT (29.11280 -11.72610)
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2020-09-01 08:42:00	-17.1728	20.0847	329.4	1.5	1.2	2020-09-01	842	Terra	MODIS	79	6.03	304.9	30.4	D	0	2020-09-01 08:42:00	POINT (20.08470 -17.17280)
2020-09-01 08:42:00	-18.4965	19.8103	324.7	1.5	1.2	2020-09-01	842	Terra	MODIS	72	6.03	306.0	20.2	D	0	2020-09-01 08:42:00	POINT (19.81030 -18.49650)
2020-09-01 08:44:00	-27.9381	24.6669	328.1	1.2	1.1	2020-09-01	844	Terra	MODIS	83	6.03	297.1	27.9	D	0	2020-09-01 08:44:00	POINT (24.66690 -27.93810)
2020-09-01 08:44:00	-27.9366	24.6552	311.5	1.2	1.1	2020-09-01	844	Terra	MODIS	62	6.03	296.1	7.7	D	0	2020-09-01 08:44:00	POINT (24.65520 -27.93660)
2020-09-01 08:44:00	-27.4436	24.7104	320.0	1.1	1.1	2020-09-01	844	Terra	MODIS	70	6.03	299.1	15.0	D	0	2020-09-01 08:44:00	POINT (24.71040 -27.44360)

898 rows × 17 columns

Now overlay the fire pixels ontop of the MODIS image, along with the FRP for each fire pixel. Try zooming in, you should be able to see clear smoke trails at the locations of some of the fires!

Show code cell source Hide code cell source

rgb = modis_dataset['true_color'].hvplot.rgb(x='longitude', y='latitude', bands='bands', data_aspect=1, hover=False, title='True Colour', rasterize=True)
points = fire_points.hvplot.points('longitude', 'latitude', c='frp', cmap='autumn', logz=True, alpha=0.4)
plot_FIRMS_MODIS_rgb = rgb*points
plot_FIRMS_MODIS_rgb

We can also overlay the fire pixels ontop of the MODIS 3.7 micron wavelength band. The 3.7 band is a thermal band. Fires will appear as very bright pixels in image. Try zooming in, you should be able to clearly see bright spots at the location of each fire pixel.

Show code cell source Hide code cell source

thermal = modis_dataset['20'].hvplot.image(x='longitude', y='latitude', cmap='viridis', data_aspect=1, hover=False, title='Band 20: 3.7 micron', rasterize=True)
points = fire_points.hvplot.points('longitude', 'latitude', c='frp', cmap='autumn', logz=True, alpha=0.4)
plot_FIRMS_MODIS_thermal = thermal*points
plot_FIRMS_MODIS_thermal

Summary

This notebook has demonstrated the use of:

• The satpy package to easily load, plot and regrid satellite data from the MODIS sensor.

• hvplot to interactively visualise wildfire pixels detected from the MODIS sensor.

• geopandas to load, filter, and manipulate historical wildfire pixel data.

Citing this Notebook

Please see CITATION.cff for the full citation information. The citation file can be exported to APA or BibTex formats (learn more here).

Additional information

Dataset: MOD021KM & FIRMS (Collection 61)

License: The code in this notebook is licensed under the MIT License. The Environmental Data Science book is licensed under the Creative Commons by Attribution 4.0 license. See further details here.

Contact: If you have any suggestion or report an issue with this notebook, feel free to create an issue or send a direct message to environmental.ds.book@gmail.com.

Notebook repository version: v1.0.5
Last tested: 2024-03-11