i.sentinel.download - GRASS GIS manual

Note: This document is for an older version of GRASS GIS that will be discontinued soon. You should upgrade, and read the current manual page.

NAME

i.sentinel.download - Downloads Sentinel satellite data from Copernicus Open Access Hub, USGS Earth Explorer, or Google Cloud Storage.

KEYWORDS

imagery, satellite, Sentinel, download

SYNOPSIS

i.sentinel.download

i.sentinel.download --help

i.sentinel.download [-plsb] [settings=name] [output=name] [footprints=name] [map=name] [area_relation=string] [clouds=integer] [producttype=string] [start=string] [end=string] [limit=integer] [query=string] [uuid=string[,string,...]] [relativeorbitnumber=integer[,integer,...]] [sleep=string] [retry=string] [datasource=string] [sort=string[,string,...]] [order=string] [--overwrite] [--help] [--verbose] [--quiet] [--ui]

Flags:

-p: Print compiled query string and exit (only supported for ESA_COAH)
-l: List filtered products and exit
-s: Skip scenes that have already been downloaded after ingestiondate
-b: Use the borders of the AOI polygon and not the region of the AOI
--overwrite: Allow output files to overwrite existing files
--help: Print usage summary
--verbose: Verbose module output
--quiet: Quiet module output
--ui: Force launching GUI dialog

Parameters:

settings=name: Full path to settings file (user, password); '-' for standard input
output=name: Name for output directory where to store downloaded Sentinel data
footprints=name: Only supported for download from ESA_Copernicus Open Access Hub; Name for output vector map with footprints
map=name: Name of input vector map to define Area of Interest (AOI); If not given then current computational extent is used
area_relation=string: ESA Copernicus Open Access Hub allows all three, USGS Earth Explorer only 'Intersects' option; Spatial relation of footprint to AOI; Options: Intersects, Contains, IsWithin; Default: Intersects
clouds=integer: Maximum cloud cover percentage for Sentinel scene
producttype=string: USGS Earth Explorer only supports S2MSI1C; Sentinel product type to filter; Options: SLC, GRD, OCN, S2MSI1C, S2MSI2A, S2MSI2Ap, S3OL1EFR, S3OL1ERR, S3OL1SPC, S3OL1RAC, S3SL1RBT, S3OL2WFR, S3OL2WRR, S3OL2LFR, S3OL2LRR, S3SL2LST, S3SL2FRP, S3SY2SYN, S3SY2VGP, S3SY2VG1, S3SY2V10, S3SY2AOD, S3SR2LAN; Default: S2MSI2A
start=string: Start date ('YYYY-MM-DD')
end=string: End date ('YYYY-MM-DD')
limit=integer: Limit number of Sentinel products
query=string: USGS Earth Explorer only supports query options "identifier", "filename" (in ESA name format) or "usgs_identifier" (in USGS name format); Extra search keywords to use in the query
uuid=string[,string,...]: List of UUID to download
relativeorbitnumber=integer[,integer,...]: _Only supported by ESA Copernicus Open Access Hub.; Relative orbit number to download (Sentinel-1: from 1 to 175; Sentinel-2: from 1 to 143; Sentinel-3: from 1 to 385)
sleep=string: Sleep time in minutes before retrying to download data from ESA LTA
retry=string: Maximum number of retries before skipping to the next scene at ESA LTA; Default: 5
datasource=string: Default is ESA Copernicus Open Access Hub (ESA_COAH), but Sentinel-2 L1C data can also be acquired from USGS Earth Explorer (USGS_EE) or Google Cloud Storage (GCS); Data-Hub to download scenes from.; Options: ESA_COAH, USGS_EE, GCS; Default: ESA_COAH
sort=string[,string,...]: Sort by values in given order; Options: ingestiondate, cloudcoverpercentage, footprint; Default: cloudcoverpercentage,ingestiondate,footprint
order=string: Sort order (see sort parameter); Options: asc, desc; Default: asc

DESCRIPTION
NOTES
- Long Term Archive (LTA)
EXAMPLES
REQUIREMENTS
SEE ALSO
AUTHORS

DESCRIPTION

The i.sentinel.download addon allows downloading Sentinel satellite products from the Copernicus Open Access Hub, USGS Earth Explorer, or Google Cloud Storage.

Copernicus Open Access Hub

Using the Copernicus Open Access Hub for searching and downloading Copernicus Sentinel data is the default option. The following product types (parameter producttype) are currently supported for download from the Copernicus Open Access Hub:

Sentinel-1 (SAR; available from Oct 2014 to present day) (SAR) products:
- SLC: Single Look Complex (Level-1)
- GRD: Ground Range Detected (Level-1)
- OCN: Ocean products for wind, wave and currents applications (Level-2)
Sentinel-2 (optical and infrared; available from July 2015 to present day) (optical) products:
- S2MSI2A: operational Bottom-Of-Atmosphere reflectances in cartographic geometry Level-2A)
- (S2MSI2Ap: pilot products of Bottom-Of-Atmosphere reflectances in cartographic geometry Level-2A), until March 2018)
- S2MSI1C: Top-Of-Atmosphere reflectances in cartographic geometry (Level-1C)
Sentinel-3 (OLCI and SLSTR instrument data products at level 2, for OLCI sensor also Level 1; available from April 2018 to present day) (optical) products:
- S3OL1EFR: Land colour and atmosphere TOA radiances at full resolution
- S3OL1ERR: Land colour and atmosphere TOA radiances at reduced resolution
- S3OL1RAC: Dark offset and gain coefficients from radiometric calibration
- S3OL1SAC: Wavelength characterisation from spectral calibration
- S3SL1RBT: Brightness temperatures and radiances
- S3OL2WFR: Ocean colour, water and atmosphere geophysical parameters
- S3OL2WRR: Ocean colour, water and atmosphere geophysical parameters at reduced resolution
- S3OL2LFR: Land colour and atmosphere geophysical parameters
- S3OL2LRR: Land colour and atmosphere geophysical parameters at reduced resolution
- S3SL2LST: Land Surface Temperature
- S3SL2FRP: Fire Radiative Power
- S3SR2LAN: Land Surface Height
- S3SY2SYN: Surface Reflectance and Aerosol parameters over Land
- S3SY2VGP: 1 km VEGETATION-Like product (~VGT-P) - TOA Reflectance
- S3SY2VG1: 1 km VEGETATION-Like product (~VGT-S1) 1 day synthesis surface reflectance and NDVI
- S3SY2V10: 1 km VEGETATION-Like product (~VGT-S10) 10 day synthesis surface reflectance and NDVI
- S3SY2AOD: Global Aerosol parameter over land and sea on super pixel resolution (4.5 km x 4.5 km)

To connect to the Copernicus Open Access Hub both a user name and password are required; see Register new account page for signing up.

USGS Earth Explorer

The following product types (parameter producttype) are currently supported for download from the USGS Earth Explorer:

Sentinel-2 (optical and infrared):
- S2MSI1C: Top-Of-Atmosphere reflectances in cartographic geometry (Level-1C)

To connect to the USGS Earth Explorer both a user name and password are required; see EROS Registration System page for signing up.

Google Cloud Storage

The following product types (parameter producttype) are currently supported for download from Google Cloud Storage:

Sentinel-2 (optical and infrared):
- S2MSI2A: operational Bottom-Of-Atmosphere reflectances in cartographic geometry Level-2A)
- S2MSI1C: Top-Of-Atmosphere reflectances in cartographic geometry (Level-1C)

Querying and filtering data on Google Cloud Storage requires the fee-based usage of BigQuery, which is currently not supported by i.sentinel.download. Instead, when using the datasource=GCS option, querying is performed using the Copernicus Open Access Hub interface, while only the download step makes use of Google Cloud Storage (download links are automatically generated from the respective Sentinel-2 identifier). As a consequence, a settings file with valid credentials for the Copernicus Open Access Hub is required for downloading from Google Cloud Storage in i.sentinel.download.

Credentials file

i.sentinel.download reads the user credentials from the settings file. The file must contain at least two lines:

myusername
mypassword

Optionally on third line custom API URL can be defined. Note that empty lines in settings file are silently skipped.

NOTES

The data hub to download from can be indicated with the datasource option. ESA's Copernicus Open Access Hub is the default option.

User credentials can be also defined interactively when settings=- is given. Note that interactive prompt does not work in the graphical user interface.

Insert username: myusername
Insert password:
Insert API URL (leave empty for https://scihub.copernicus.eu/apihub):

By default Sentinel products are sorted by cloudcoverpercentage and ingestiondate (see sort option). By default, only products which footprint intersects current computation region extent (area of interest, AOI) are filtered. The AOI can be optionally defined also by vector map. In addition the spatial relation between AOI and the footprint (area_relation) can be set to

Contains: to only return scenes where the AOI is contained inside the footprint (Only supported by Copernicus Open Access Hub)
IsWithin: to only return scenes where the footprint is contained inside the AOI (Only supported by Copernicus Open Access Hub)
Intersects: to return all scenes where the footprint intersects the AOI (default)

Filtered products can be reduced by limit option.

i.sentinel.download limits the default search for products to the last 60 days; an exact date range can be defined by start and end parameters to search beyond that.

Sentinel products can be also filtered by producttype or, in case of Sentinel-2 data, maximum clouds cover percentage.

Extra search keywords can be specified with query. Multiple keywords can be listed separated with comma (e.g. 'polarisationmode=VV,orbitdirection=Ascending'). Note that ESA maintains a list of valid search keywords that can be used with query. Usage of the query option for searching and downloading from the USGS Earth Explorer allows the keywords identifier and filename, both in ESA format. Additionally, the keyword usgs_identifier can be used to search for a specific scene in USGS name format (see USGS Entity ID)

i.sentinel.download also allows downloading of Sentinel products by specifying a (list of) UUID. This operation is performed by the uuid option. Note that this option is mutually exclusive with all other filtering options. The format of UUID differs between the Copernicus Open Access Hub and the USGS Earth Explorer (see examples below).

In case a Sentinel data download was interrupted, i.sentinel.download will continue to download once the command is restarted and the partially downloaded file (with .incomplete file extention) still present.

The output directory is created if not yet available.

The query string sent to ESA can be shown with --verbose.

Long Term Archive (LTA)

Access to the product URL for data that are no longer available online in the Copernicus Open Access Hub will automatically trigger the retrieval of the data from the LTA, the actual download may be initiated once the data are restored (within minutes to hours). Full details are available in the updated user guide, section Long Term Archive (LTA) Access. In this issue of sentinelsat, it is stated (as of 2020):
Please be advised that a user quota on the maximum number of requests per hour per user is set. Note that if you

retrieve data directly from the GUI, you can retrieve only 1 product per half hour while
from APIHub 20 products each 12h

To work with LTA you need to define the sleep parameter (and you can set the maximum number of retries with the retry parameter) to retry a download after the sleep value in minutes.

As an alternative to triggering re-upload Sentinel-2 data from the LTA, scenes can be downloaded from USGS Earth Explorer or Google Cloud Storage where all data remain online and directly accessible. However, this applies only to Sentinel-2 data in levels 1C (USGS and GCS) and 2A (only GCS).

EXAMPLES

List filtered products

Find all atmospherically corrected Sentinel-2 L2A products (S2MSI2) in 2018 (area in Italy as an example):

g.region n=42 w=12 s=41 e=13 res=0:01 -p

i.sentinel.download -l settings=credentials.txt producttype=S2MSI2A start=2018-01-01 end=2018-12-31

357 Sentinel product(s) found
379e2896-408d-4642-bc6f-ee941e395c32 2018-04-19T10:10:31Z  0% S2MSI2A
0e3e361d-e8e0-4e71-a10d-c1faf1640beb 2018-04-19T10:10:31Z  0% S2MSI2A
72ed25c8-5a36-40a5-97ce-e0b4d6347c7f 2018-04-06T10:00:31Z  0% S2MSI2A
eee28af8-aaa6-4e59-9902-dd5cbfc08bab 2018-04-06T10:00:31Z  0% S2MSI2A
e2a9d80d-da35-44bd-ab88-76bad398da41 2018-07-18T10:10:31Z  0% S2MSI2A
...

Sort products by ingestiondate, limit cloud coverage to 3% per scene:

g.region n=42 w=12 s=41 e=13 res=0:01 -p
i.sentinel.download -l settings=credentials.txt producttype=S2MSI2A start=2018-01-01 end=2018-12-31 sort=ingestiondate order=desc clouds=3

191 Sentinel product(s) found
1b0ee27e-c51a-49d9-b2a4-78db128a49f9 2018-12-27T10:04:19Z  0% S2MSI2A
8da04aa9-5d8b-42bc-abb3-d2c9b61c5ee4 2018-12-27T10:04:19Z  1% S2MSI2A
e4f324f4-8e98-440c-9141-ae3d24cd1faa 2018-12-27T10:04:19Z  1% S2MSI2A
be507500-8538-44f0-969c-d6d45a16beea 2018-12-27T10:04:19Z  0% S2MSI2A
1ab5dd20-4845-4325-bd71-5d0596f4d11d 2018-12-27T10:04:19Z  0% S2MSI2A
...

Create a vector map of footprints of S-2 scenes with clouds limited to 3% per scene (note that topological errors will be shown since some footprint overlap):

g.region n=42 w=12 s=41 e=13 res=0:01 -p
i.sentinel.download -l settings=credentials.txt producttype=S2MSI2A start=2018-01-01 end=2018-12-31 clouds=1 footprints=s2_scenes_footprints

131 Sentinel product(s) found
Writing footprints into <s2_scenes_footprints>...
...
379e2896-408d-4642-bc6f-ee941e395c32 2018-04-19T10:10:31Z  0% S2MSI2A
0e3e361d-e8e0-4e71-a10d-c1faf1640beb 2018-04-19T10:10:31Z  0% S2MSI2A
...

Find Sentinel-2 L1C products (S2MSI1C) of last 60 days (default) covering current computation region extent:

g.region n=42 w=12 s=41 e=13 res=0:01 -p
i.sentinel.download -l settings=credentials.txt producttype=S2MSI1C sort=ingestiondate

57 Sentinel product(s) found
c5f6e871-3cd9-471f-a022-712f5e10a43a 2018-11-10T10:12:39Z 15% S2MSI1C
a83ad139-c420-4433-bd21-f7ed35536e69 2018-11-10T10:12:39Z 14% S2MSI1C
6a4f093a-3fa7-466e-8ec9-77718d7b8eac 2018-11-12T10:02:41Z  8% S2MSI1C
9a668bfa-1a7e-4d92-8c51-ed2024415e6a 2018-11-15T10:12:51Z  6% S2MSI1C
5f3d19a2-eddb-42eb-ab6d-67f0fdd18c58 2018-11-15T10:12:51Z  5% S2MSI1C
...

Find Sentinel-1 products by one or several specified UUIDs:

i.sentinel.download -l settings=credentials.txt uuid=a45d5bbe-6919-49fc-84fd-72a7634d2035
a45d5bbe-6919-49fc-84fd-72a7634d2035 2018-06-04T05:26:01Z cloudcover_NA SLC

Find Sentinel-1 products with date range filter and query filter limit to a specific polarisation mode (see here for query details):

i.sentinel.download -l settings=credentials.txt producttype=SLC start=2018-01-01 end=2018-12-31 query='polarisationmode=VV'
34 Sentinel product(s) found
047db83e-db60-45f3-a6d2-3a32588dac01 2018-01-10T23:14:05Z cloudcover_NA SLC
fc6107a5-9daf-45cc-a40a-0e5502c464b6 2018-01-22T23:14:05Z cloudcover_NA SLC
4ad9b650-2954-4ff5-977b-90aedbed6514 2018-02-03T23:14:05Z cloudcover_NA SLC
[...]

Find Sentinel-2 L1C products (S2MSI1C) covering an exemplary region in Germany with temporal and cloud filter from the USGS Earth Explorer. Note the different formats of UUID and identifier used by the USGS Earth Explorer (first two result columns):

g.region n=51 w=6 s=50 e=7 res=0:01 -p

i.sentinel.download datasource=USGS_EE start=2017-09-01 end=2017-12-01 clouds=10 producttype=S2MSI1C settings=credentials.txt -l
20 Sentinel product(s) found
2177169 L1C_T31UGR_A011488_20170903T105515 2017-09-03  0%
2115539 L1C_T31UGR_A011631_20170913T105335 2017-09-13  0%
2184043 L1C_T31UGR_A011774_20170923T105022 2017-09-23  0%
2228254 L1C_T31UGR_A011917_20171003T105024 2017-10-03  0%
2277543 L1C_T31UGR_A012060_20171013T105315 2017-10-13  0%
5594498 L1C_T32ULB_A003180_20171015T104525 2017-10-15  0%
[...]

Find Sentinel-2 L1C products (S2MSI1C) with specific UUIDs from the USGS Earth Explorer:

i.sentinel.download datasource=USGS_EE uuid=2177169,2115539 settings=credentials.txt -l
2 Sentinel product(s) found
2177169 L1C_T31UGR_A011488_20170903T105515 2017-09-03  0%
2115539 L1C_T31UGR_A011631_20170913T105335 2017-09-13  0%

Download Sentinel products

Download first (limit=1) S2MSI2A product found into the data directory:

g.region n=42 w=12 s=41 e=13 res=0:01 -p
i.sentinel.download settings=credentials.txt producttype=S2MSI2A start=2018-05-01 end=2018-05-31 limit=1 output=s2_L2A_may2018/

Download first (limit=1) S2MSI2A product found from Google Cloud Storage:

i.sentinel.download settings=credentials.txt producttype=S2MSI2A start=2018-05-01 end=2018-05-31 limit=1 output=s2_L2A_may2018/ datasource=GCS

The downloaded Sentinel data can subsequently be easily imported into GRASS GIS using i.sentinel.import module.

Download Sentinel products by UUID

Example of downloading a single Sentinel product by UUID:

i.sentinel.download settings=credentials.txt uuid=6a10da0f-7777-4818-bc2c-4680e82297ac output=s2_data/

Example of downloading a single Sentinel product by UUID from Google Cloud Storage

i.sentinel.download datasource=GCS settings=credentials.txt uuid=6a10da0f-7777-4818-bc2c-4680e82297ac output=s2_data/

Example of downloading a single Sentinel product by UUID from the USGS Earth Explorer:

i.sentinel.download datasource=USGS_EE settings=credentials.txt uuid=2177169 output=s2_data/

Download Sentinel products by scene name

i.sentinel.download also allows downloading of Sentinel products by specifying a (list of) scene names (name of the SAFE.zip file). This operation is performed by the query option. Note that this option is mutually exclusive with all other filtering options.

Example of downloading a single Sentinel product by scene names (name of the SAFE.zip file). Importantly, the producttype, start and end date need to be specified to override the default settings (otherwise no product will be found). There are two query variants, either with identifier or filename:

# North Carolina sample dataset example

# Sentinel-1, identifier= without ".SAFE" in the name
i.sentinel.download settings=credentials.txt \
  query='identifier=S1A_IW_GRDH_1SDV_20210130T231425_20210130T231450_036374_0444CC_0EA9' \
  producttype=GRD start=2021-01-01 end=2021-02-15 output=s1_data/

# Sentinel-1, filename= requires ".SAFE" in the name
i.sentinel.download settings=credentials.txt \
  query='filename=S1A_IW_GRDH_1SDV_20210130T231425_20210130T231450_036374_0444CC_0EA9.SAFE' \
  producttype=GRD start=2021-01-01 end=2021-02-15 output=s1_data/


# Sentinel-2, identifier= without ".SAFE" in the name
i.sentinel.download settings=credentials.txt \
  query='identifier=S2B_MSIL2A_20210120T155559_N0214_R054_T17SPV_20210120T201821' \
  producttype=S2MSI2A start=2021-01-01 end=2021-02-15 output=s2_data/

# Sentinel-2, filename= requires ".SAFE" in the name
i.sentinel.download settings=credentials.txt \
  query='filename=S2B_MSIL2A_20210120T155559_N0214_R054_T17SPV_20210120T201821.SAFE' \
  producttype=S2MSI2A start=2021-01-01 end=2021-02-15 output=s2_data/

Example of downloading a single Sentinel product by scene name from Google Cloud Storage:

i.sentinel.download datasource=GCS start=2021-01-01 end=2021-02-15 producttype=S2MSI2A \
  query="filename=S2B_MSIL2A_20210120T155559_N0214_R054_T17SPV_20210120T201821.SAFE" settings=credentials.txt \
  output=s2_data/

Example of downloading a single Sentinel product by identifier from Google Cloud Storage:

  # in this case, no settings file is required, because the identifier can be directly used to create a GCS download URL
  i.sentinel.download datasource=GCS query="identifier=S2B_MSIL2A_20210120T155559_N0214_R054_T17SPV_20210120T201821" \
    output=s2_data/

Example of downloading a single Sentinel product from USGS Earth Explorer by the USGS-identifier:

i.sentinel.download datasource=USGS_EE start=2017-09-01 end=2017-12-01 producttype=S2MSI1C \
  query="usgs_identifier=L1C_T32ULA_A003180_20171015T104525" settings=credentials.txt \
  output=s2_data/

It is also possible to use the keywords filename/identifier in ESA formatting in query when downloading from the USGS Earth Explorer:

i.sentinel.download datasource=USGS_EE start=2017-10-01 end=2017-11-01 producttype=S2MSI1C \
  query='filename=S2B_MSIL1C_20171015T104009_N0205_R008_T32ULA_20171015T104525' settings=credentials.txt \
  output=s2_data/

REQUIREMENTS

Sentinelsat library
Pandas library
landsatxplore library

AUTHORS

Martin Landa, GeoForAll Lab, CTU in Prague, Czech Republic with support of OpenGeoLabs company
Guido Riembauer, mundialis (USGS and GCS provider support)

SOURCE CODE

Available at: i.sentinel.download source code (history)

Latest change: Sunday Jun 02 17:36:01 2024 in commit: 6abf6944be8233ca26531f3fd4d9afa849fa4914