API

Getting started

This section gives a quick overview of the API. You can also take a look at the tutorials.

Initialisation

Initialize() provides a shortcut for initialising the Earth Engine API.

import geedim as gd
gd.Initialize()

Searching image collections

Any Earth Engine image collection can be searched with MaskedCollection. Here, we search for Landsat-8 surface reflectance images over Stellenbosch, South Africa.

# geojson search polygon
region = {
    'type': 'Polygon', 'coordinates': [[
        (19, -34), (19, -33.8), (18.8, -33.8), (18.8, -34), (19., -34)
    ]]
}

# create & search a landsat-8 collection, reporting cloudless portions
coll = gd.MaskedCollection.from_name('LANDSAT/LC08/C02/T1_L2')
filt_coll = coll.search('2019-01-01', '2019-02-15', region, cloudless_portion=0)

# display the search results
print(filt_coll.schema_table)
print(filt_coll.properties_table)

The output:

ABBREV    DESCRIPTION
--------- ---------------------------------------
ID        Earth Engine image id
DATE      Image capture date/time (UTC)
FILL      Portion of region pixels that are valid (%)
CLOUDLESS Portion of filled pixels that are cloud/shadow free (%)
GRMSE     Orthorectification RMSE (m)
SAA       Solar azimuth angle (deg)
SEA       Solar elevation angle (deg)

ID                                          DATE              FILL CLOUDLESS GRMSE   SAA   SEA
------------------------------------------- ---------------- ----- --------- ----- ----- -----
LANDSAT/LC08/C02/T1_L2/LC08_175083_20190101 2019-01-01 08:35 99.83     55.62  8.48 79.36 59.20
LANDSAT/LC08/C02/T1_L2/LC08_175084_20190101 2019-01-01 08:35 99.79     60.35  9.71 77.28 58.67
LANDSAT/LC08/C02/T1_L2/LC08_175083_20190117 2019-01-17 08:35 99.98     94.90  8.84 76.98 56.79
LANDSAT/LC08/C02/T1_L2/LC08_175084_20190117 2019-01-17 08:35 99.97     95.07  9.75 75.13 56.21
LANDSAT/LC08/C02/T1_L2/LC08_175083_20190202 2019-02-02 08:34 99.91     95.82  8.46 71.91 54.00
LANDSAT/LC08/C02/T1_L2/LC08_175084_20190202 2019-02-02 08:35 99.87     95.21  9.21 70.34 53.30

Image creation and download

Images can be created, masked and downloaded with the MaskedImage class. Typically, one would pass the Earth Engine image ID to MaskedImage.from_id() to create the image.

# create a landsat-8 image from its ID
im = gd.MaskedImage.from_id(
    'LANDSAT/LC08/C02/T1_L2/LC08_175083_20190117', mask=False)

# download a region of the image with 'average' resampling to 60m pixels, and
# data type conversion to 16 bit unsigned int
im.download('landsat_8_image.tif', region=region, resampling='average',
            scale=60, dtype='uint16')

Compositing

Let’s form a cloud/shadow-free composite of the search result images, using the q-mosaic method, then download the result. By specifying the region parameter to MaskedCollection.composite(), we prioritise selection of pixels from the least cloudy images when forming the composite.

Note

When downloading composite images, the region, crs and scale parameters must be specified, as the image has no fixed (known) projection.

# find a 'q-mosaic' composite image of the search result images, prioritising
# the least cloudy image by specifying `region`
comp_im = filt_coll.composite(method='q-mosaic', region=region)
# download the composite, specifying crs, region, and scale
comp_im.download('landsat_8_comp_image.tif', region=region, crs='EPSG:32634',
                 scale=30)

Cloud/shadow masking

All MaskedImage and MaskedCollection methods that involve cloud/shadow masking (MaskedImage.from_id(), MaskedCollection.search(), and MaskedCollection.composite()) take optional cloud/shadow masking **kwargs. See MaskedImage.__init__() for a description of these parameters.

Here, we create and download a cloud/shadow masked Sentinel-2 image, specifying a cloud probability threshold of 30%.

# create a cloud/shadow masked Sentinel-2 image, specifying a cloud
# probability threshold of 30%
im = gd.MaskedImage.from_id(
    'COPERNICUS/S2_SR/20190101T082331_20190101T084846_T34HCH', mask=True, prob=30)
# download a region of the masked image, downsampling to 20m pixels
im.download('s2_sr_image.tif', region=region, scale=20, resampling='average')

Image metadata

geedim populates downloaded files with metadata from the source Earth Engine image, and the associated STAC entry. The next code snippet uses rasterio to read the metadata of the downloaded Sentinel-2 image.

import rasterio as rio
with rio.open('s2_sr_image.tif', 'r') as ds:
    print('Image properties:\n', ds.tags())
    print('Band names:\n', ds.descriptions)
    print('Band 1 properties:\n', ds.tags(1))

Output:

Image properties:
 {'AOT_RETRIEVAL_ACCURACY': '0', 'AREA_OR_POINT': 'Area', 'CLOUDY_PIXEL_PERCENTAGE': '4.228252', 'CLOUD_COVERAGE_ASSESSMENT': '4.228252', 'CLOUD_SHADOW_PERCENTAGE': '0.353758', 'DARK_FEATURES_PERCENTAGE': '1.390344', 'DATASTRIP_ID': 'S2A_OPER_MSI_L2A_DS_MTI__20190101T112242_S20190101T084846_N02.11', 'DATATAKE_IDENTIFIER': 'GS2A_20190101T082331_018422_N02.11', 'DATATAKE_TYPE': 'INS-NOBS', 'DEGRADED_MSI_DATA_PERCENTAGE': '0', 'FORMAT_CORRECTNESS': 'PASSED', 'GENERAL_QUALITY': 'PASSED', 'GENERATION_TIME': '1546341762000', 'GEOMETRIC_QUALITY': 'PASSED', 'GRANULE_ID': 'L2A_T34HCH_A018422_20190101T084846', 'HIGH_PROBA_CLOUDS_PERCENTAGE': '1.860266', 'LICENSE': 'https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR#terms-of-use', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B1': '197.927117994', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B10': '200.473257333', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B11': '199.510962726', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B12': '198.925728126', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B2': '204.233801024', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B3': '201.623624653', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B4': '200.124411228', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B5': '199.531415295', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B6': '199.06932777', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B7': '198.686746475', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B8': '202.762429499', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B8A': '198.389627535', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B9': '197.722038042', 'MEAN_INCIDENCE_ZENITH_ANGLE_B1': '3.18832352559', 'MEAN_INCIDENCE_ZENITH_ANGLE_B10': '2.74480253282', 'MEAN_INCIDENCE_ZENITH_ANGLE_B11': '2.9240632909', 'MEAN_INCIDENCE_ZENITH_ANGLE_B12': '3.12457836272', 'MEAN_INCIDENCE_ZENITH_ANGLE_B2': '2.56258512587', 'MEAN_INCIDENCE_ZENITH_ANGLE_B3': '2.66821142821', 'MEAN_INCIDENCE_ZENITH_ANGLE_B4': '2.78791939543', 'MEAN_INCIDENCE_ZENITH_ANGLE_B5': '2.86049380258', 'MEAN_INCIDENCE_ZENITH_ANGLE_B6': '2.93757718579', 'MEAN_INCIDENCE_ZENITH_ANGLE_B7': '3.01912758709', 'MEAN_INCIDENCE_ZENITH_ANGLE_B8': '2.61179829178', 'MEAN_INCIDENCE_ZENITH_ANGLE_B8A': '3.10418395274', 'MEAN_INCIDENCE_ZENITH_ANGLE_B9': '3.28253454154', 'MEAN_SOLAR_AZIMUTH_ANGLE': '74.331216318', 'MEAN_SOLAR_ZENITH_ANGLE': '27.589988524', 'MEDIUM_PROBA_CLOUDS_PERCENTAGE': '0.774948', 'MGRS_TILE': '34HCH', 'NODATA_PIXEL_PERCENTAGE': '2.7e-05', 'NOT_VEGETATED_PERCENTAGE': '72.305781', 'PROCESSING_BASELINE': '02.11', 'PRODUCT_ID': 'S2A_MSIL2A_20190101T082331_N0211_R121_T34HCH_20190101T112242', 'RADIATIVE_TRANSFER_ACCURACY': '0', 'RADIOMETRIC_QUALITY': 'PASSED', 'REFLECTANCE_CONVERSION_CORRECTION': '1.03413456106', 'SATURATED_DEFECTIVE_PIXEL_PERCENTAGE': '0', 'SENSING_ORBIT_DIRECTION': 'DESCENDING', 'SENSING_ORBIT_NUMBER': '121', 'SENSOR_QUALITY': 'PASSED', 'SNOW_ICE_PERCENTAGE': '0.0156', 'SOLAR_IRRADIANCE_B1': '1884.69', 'SOLAR_IRRADIANCE_B10': '367.15', 'SOLAR_IRRADIANCE_B11': '245.59', 'SOLAR_IRRADIANCE_B12': '85.25', 'SOLAR_IRRADIANCE_B2': '1959.72', 'SOLAR_IRRADIANCE_B3': '1823.24', 'SOLAR_IRRADIANCE_B4': '1512.06', 'SOLAR_IRRADIANCE_B5': '1424.64', 'SOLAR_IRRADIANCE_B6': '1287.61', 'SOLAR_IRRADIANCE_B7': '1162.08', 'SOLAR_IRRADIANCE_B8': '1041.63', 'SOLAR_IRRADIANCE_B8A': '955.32', 'SOLAR_IRRADIANCE_B9': '812.92', 'SPACECRAFT_NAME': 'Sentinel-2A', 'system-asset_size': '1820790758', 'system-index': '20190101T082331_20190101T084846_T34HCH', 'system-time_end': '1546332584000', 'system-time_start': '1546332584000', 'THIN_CIRRUS_PERCENTAGE': '1.593038', 'UNCLASSIFIED_PERCENTAGE': '1.202621', 'VEGETATION_PERCENTAGE': '18.241563', 'WATER_PERCENTAGE': '2.262083', 'WATER_VAPOUR_RETRIEVAL_ACCURACY': '0'}
Band names:
 ('B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B8A', 'B9', 'B11', 'B12', 'AOT', 'WVP', 'SCL', 'TCI_R', 'TCI_G', 'TCI_B', 'MSK_CLDPRB', 'MSK_SNWPRB', 'QA10', 'QA20', 'QA60', 'FILL_MASK', 'CLOUD_MASK', 'CLOUDLESS_MASK', 'SHADOW_MASK', 'CLOUD_PROB', 'CLOUD_DIST')
Band 1 properties:
 {'center_wavelength': '0.4439', 'description': 'Aerosols', 'gsd': '60', 'name': 'B1', 'scale': '0.0001', 'wavelength': '443.9nm (S2A) / 442.3nm (S2B)'}

Computed images and user memory

Earth engine has a size limit of 32 MB on download requests. geedim avoids exceeding this by tiling downloads. However, Earth engine also has a limit on user memory for image computations. This limit can be exceeded when downloading large computed images (such as custom user images or geedim generated composites), raising a user memory limit exceeded error. Unfortunately, there is no way for geedim to adjust tiles to avoid exceeding this limit, as the memory requirements of a computation are not known in advance. The user has two options for working around this error:

1) max_tile_size

Decreasing the max_tile_size argument to geedim.mask.MaskedImage.download() reduces the user memory required by computations. The user would need to experiment to find a reduced value that solves any memory limit problem. --max-tile-size is the equivalent option on the command line.

import ee
# create a computed ee.Image
ee_im = ee.Image('COPERNICUS/S2_SR/20190101T082331_20190101T084846_T34HCH')
comp_im = ee_im.select('B3').entropy(ee.Kernel.square(5))

# encapsulate in MaskedImage, and download with max_tile_size=8
im = gd.MaskedImage(comp_im)
im.download('s2_entropy.tif', region=region, max_tile_size=8)

2) Exporting

Exporting the image to an Earth Engine asset, and then downloading. Exporting images is not subject to the user memory limit, and once exported, computation on the asset image is complete. The exported asset image can then be downloaded in the standard way.

# create EE asset ID & export computed image to asset
asset_id = f'projects/<your cloud project>/assets/s2_entropy'
_ = im.export(asset_id, type='asset', region=region, wait=True)

# create and download the asset image
im = gd.MaskedImage.from_id(asset_id)
im.download('s2_entropy.tif')

Reference

MaskedImage

class geedim.mask.MaskedImage

__init__(ee_image: Image, mask: bool = False, region: dict = None, **kwargs)

A class for describing, masking and downloading an Earth Engine image.

Parameters:

ee_image (ee.Image) – Earth Engine image to encapsulate.
mask (bool, optional) – Whether to mask the image.
region (dict, optional) – A geojson polygon inside of which to find statistics for the image. These values are stored in the image properties. If None, statistics are not found (the default).
**kwargs – Cloud/shadow masking parameters - see below:
mask_cirrus (bool, optional) – Whether to mask cirrus clouds. Valid for Landsat 8-9 images, and for Sentinel-2 images with the qa mask_method.
mask_shadows (bool, optional) – Whether to mask cloud shadows.
mask_method (CloudMaskMethod, str, optional) – Method used to mask clouds. Valid for Sentinel-2 images. See CloudMaskMethod for details.
prob (float, optional) – Cloud probability threshold (%). Valid for Sentinel-2 images with the cloud-prob mask-method.
dark (float, optional) – NIR threshold [0-1]. NIR values below this threshold are potential cloud shadows. Valid for Sentinel-2 images.
shadow_dist (int, optional) – Maximum distance (m) to look for cloud shadows from cloud edges. Valid for Sentinel-2 images.
buffer (int, optional) – Distance (m) to dilate cloud/shadow. Valid for Sentinel-2 images.
cdi_thresh (float, optional) – Cloud Displacement Index threshold. Values below this threshold are considered potential clouds. If this parameter is not specified (=None), the index is not used. Valid for Sentinel-2 images. See https://developers.google.com/earth-engine/apidocs/ee-algorithms-sentinel2-cdi for details.
max_cloud_dist (int, optional) – Maximum distance (m) to look for clouds when forming the ‘cloud distance’ band. Valid for Sentinel-2 images.

__new__(**kwargs)

Methods

`from_id`(image_id, **kwargs)	Create a MaskedImage, or sub-class, instance from an Earth Engine image ID.
`mask_clouds`()	Apply the cloud/shadow mask if supported, otherwise apply the fill mask.
`download`(filename[, overwrite, num_threads, ...])	Download the encapsulated image to a GeoTiff file.
`export`(filename[, type, folder, wait])	Export the encapsulated image to Google Drive, Earth Engine asset or Google Cloud Storage.
`monitor_export`(task[, label])	Monitor and display the progress of an export task.

Attributes

`ee_image`	Encapsulated Earth Engine image.
`id`	Earth Engine image ID.
`date`	Image capture date & time.
`crs`	Image CRS corresponding to minimum scale band, as an EPSG string.
`scale`	Minimum scale (m) of image bands.
`footprint`	Geojson polygon of the image extent.
`transform`	Geo-transform of the minimum scale band.
`shape`	(row, column) pixel dimensions of the minimum scale band.
`count`	Number of image bands.
`dtype`	Minimum size data type able to represent the image values.
`size`	Image size (bytes).
`has_fixed_projection`	True if the image has a fixed projection, otherwise False.
`name`	Image name (the `id` with slashes replaced by dashes).
`properties`	Earth Engine image properties.
`band_properties`	Merged STAC and Earth Engine band properties.

MaskedCollection

class geedim.collection.MaskedCollection

__init__(ee_collection: ImageCollection, add_props: List[str] = None)

A class for describing, searching and compositing an Earth Engine image collection, with support for cloud/shadow masking.

Parameters:

ee_collection (ee.ImageCollection) – Earth Engine image collection to encapsulate.
add_props (list of str, optional) – Additional Earth Engine image properties to include in properties.

__new__(**kwargs)

Methods

`from_name`(name[, add_props])	Create a MaskedCollection instance from an Earth Engine image collection name.
`from_list`(image_list[, add_props])	Create a MaskedCollection instance from a list of Earth Engine image ID strings, `ee.Image` instances and/or `MaskedImage` instances.
`search`([start_date, end_date, region, ...])	Search for images based on date, region, filled/cloudless portion, and custom criteria.
`composite`([method, mask, resampling, date, ...])	Create a composite image from the encapsulated image collection.

Attributes

`ee_collection`	Earth Engine image collection.
`name`	Name of the encapsulated Earth Engine image collection.
`image_type`	`MaskedImage` class or sub-class corresponding to images in `ee_collection`.
`properties`	A dictionary of properties for each image in the collection.
`properties_table`	`properties` formatted as a printable table string.
`schema`	A dictionary of abbreviations and descriptions for `properties`.
`schema_table`	`schema` formatted as a printable table string.
`refl_bands`	List of spectral / reflectance band names, if any.

enums

CompositeMethod

class geedim.enums.CompositeMethod

Enumeration for the compositing method, i.e. the method for finding a composite pixel from the stack of corresponding input image pixels.

q_mosaic = 'q-mosaic': Use the unmasked pixel with the highest cloud distance (distance to nearest cloud). Where more than one pixel has the same cloud distance, the first one in the stack is selected.

mosaic = 'mosaic': Use the first unmasked pixel in the stack.

medoid = 'medoid': Use the medoid of the unmasked pixels. This is the pixel from the image having the minimum sum of spectral distances to the rest of the images. Maintains the original relationship between bands. See https://www.mdpi.com/2072-4292/5/12/6481 for detail.

median = 'median': Use the median of the unmasked pixels.

mode = 'mode': Use the mode of the unmasked pixels.

mean = 'mean': Use the mean of the unmasked pixels.

__new__(value)

CloudMaskMethod

class geedim.enums.CloudMaskMethod

Enumeration for the Sentinel-2 cloud masking method.

cloud_prob = 'cloud-prob': Threshold the corresponding image from the Sentinel-2 cloud probability collection.

qa = 'qa': Use the QA60 quality assessment band.

__new__(value)

ResamplingMethod

class geedim.enums.ResamplingMethod

Enumeration for the resampling method.

near = 'near': Nearest neighbour.

bilinear = 'bilinear': Bilinear.

bicubic = 'bicubic': Bicubic.

__new__(value)

average = 'average': Average (recommended for downsampling).

ExportType

class geedim.enums.ExportType

Enumeration for the export type.

__new__(value)

drive = 'drive': Export to Google Drive.

asset = 'asset': Export to an Earth Engine asset.

cloud = 'cloud': Export to Google Cloud Storage.

Initialize

geedim.utils.Initialize(opt_url: str | None = 'https://earthengine-highvolume.googleapis.com', **kwargs)

Initialise Earth Engine.

Credentials will be read from the EE_SERVICE_ACC_PRIVATE_KEY environment variable if it exists (useful for integrating with e.g. GitHub actions).

Note

Earth Engine recommends using the high volume endpoint for applications like geedim. See the docs for more information.

Parameters:

opt_url (str) – The Earth Engine endpoint to use. If None, the default is used.
kwargs – Optional arguments to pass to ee.Initialize.

API

Getting started

Initialisation

Searching image collections

Image creation and download

Compositing

Cloud/shadow masking

Image metadata

Computed images and user memory

1) max_tile_size

2) Exporting

Reference

MaskedImage

MaskedCollection

enums

CompositeMethod

CloudMaskMethod

ResamplingMethod

ExportType

Initialize

Tutorials