API¶
Initialisation¶
Geedim extends the GEE API with the
ee.Image.gd and ee.ImageCollection.gd accessors. To enable the accessors, Geedim must be imported:
import ee
import geedim # noqa: F401
ee.Initialize()
If you use a linter, you may need to include the # noqa: F401 suppression comment after the geedim import to prevent it being removed.
Cloud masking¶
Cloud masking is supported on Landsat 4-9 collection 2 images and Sentinel-2 TOA and surface reflectance images. The cloudSupport property is True on images / collections with cloud mask support:
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
im.gd.cloudSupport
True
Image and collection accessors share the same interface for cloud masking. Masks and related bands can be added with addMaskBands(), and cloud masks applied with maskClouds():
# add mask bands created with a threshold of 0.7 on the 'cs_cdf' Cloud Score+ band
im = im.gd.addMaskBands(score=0.7, cs_band='cs_cdf')
print(im.gd.bandNames)
im = im.gd.maskClouds()
['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B8A', 'B9', 'B11', 'B12', 'AOT', 'WVP', 'SCL', 'TCI_R', 'TCI_G', 'TCI_B', 'MSK_CLDPRB', 'MSK_SNWPRB', 'QA10', 'QA20', 'QA60', 'MSK_CLASSI_OPAQUE', 'MSK_CLASSI_CIRRUS', 'MSK_CLASSI_SNOW_ICE', 'CLOUD_SCORE', 'FILL_MASK', 'CLOUDLESS_MASK', 'CLOUD_DIST']
The addMaskBands() reference documents the masking parameters for images / collections.
Filtering¶
Collections can be filtered on the filled or cloud-free portions of a given region, and other criteria, with filter():
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
# filter by date range, region bounds, and a lower limit of 60% on the cloud-free
# portion of region
filt_coll = coll.gd.filter(
'2021-10-01', '2022-04-01', region=region, cloudless_portion=60
)
The schemaTable and propertiesTable properties allow the collection contents to be displayed. schemaPropertyNames defines a set of image properties to include in the tables:
# include the VEGETATION_PERCENTAGE property in schemaTable & propertiesTable
filt_coll.gd.schemaPropertyNames += ('VEGETATION_PERCENTAGE',)
print(filt_coll.gd.schemaTable)
print(filt_coll.gd.propertiesTable)
ABBREV NAME DESCRIPTION
--------- ------------------------------- ------------------------------------------------
INDEX system:index Earth Engine image index
DATE system:time_start Image capture date/time (UTC)
FILL FILL_PORTION Portion of region pixels that are valid (%)
CLOUDLESS CLOUDLESS_PORTION Portion of filled pixels that are cloud-free (%)
RADQ RADIOMETRIC_QUALITY Radiometric quality check
GEOMQ GEOMETRIC_QUALITY Geometric quality check
SAA MEAN_SOLAR_AZIMUTH_ANGLE Solar azimuth angle (deg)
SZA MEAN_SOLAR_ZENITH_ANGLE Solar zenith angle (deg)
VAA MEAN_INCIDENCE_AZIMUTH_ANGLE_B1 View (B1) azimuth angle (deg)
VZA MEAN_INCIDENCE_ZENITH_ANGLE_B1 View (B1) zenith angle (deg)
VP VEGETATION_PERCENTAGE Percentage of pixels classified as vegetation
INDEX DATE FILL CLOUDLESS RADQ GEOMQ SAA SZA VAA VZA VP
-------------------------------------- ---------------- ------ --------- ------ ------ ----- ----- ------ ---- -----
20211006T075809_20211006T082043_T35HKC 2021-10-06 08:29 100.00 99.33 PASSED PASSED 44.88 37.00 183.01 3.06 22.25
20211021T075951_20211021T082750_T35HKC 2021-10-21 08:29 100.00 100.00 PASSED PASSED 49.71 32.07 183.15 3.07 14.52
20211031T080051_20211031T082748_T35HKC 2021-10-31 08:29 100.00 100.00 PASSED PASSED 53.80 29.38 182.67 3.07 23.20
20211115T080109_20211115T082145_T35HKC 2021-11-15 08:29 100.00 90.59 PASSED PASSED 60.95 26.59 181.27 3.06 7.68
20211130T080301_20211130T082811_T35HKC 2021-11-30 08:29 100.00 67.26 PASSED PASSED 67.94 25.46 180.43 3.06 14.30
20211210T080321_20211210T082814_T35HKC 2021-12-10 08:29 100.00 73.88 PASSED PASSED 71.57 25.60 182.17 3.06 16.56
20211220T080341_20211220T082827_T35HKC 2021-12-20 08:29 100.00 81.07 PASSED PASSED 73.81 26.29 183.00 3.07 21.30
20211225T080239_20211225T082126_T35HKC 2021-12-25 08:29 99.83 85.90 PASSED PASSED 74.37 26.83 182.86 3.06 7.51
20220109T080321_20220109T082806_T35HKC 2022-01-09 08:29 100.00 100.00 PASSED PASSED 73.66 28.85 182.17 3.06 28.63
20220119T080241_20220119T082733_T35HKC 2022-01-19 08:29 100.00 100.00 PASSED PASSED 71.52 30.51 180.43 3.06 18.67
20220124T080119_20220124T081814_T35HKC 2022-01-24 08:29 100.00 100.00 PASSED PASSED 70.05 31.42 179.96 3.06 13.94
20220129T080201_20220129T082905_T35HKC 2022-01-29 08:29 100.00 100.00 PASSED PASSED 68.32 32.34 180.45 3.06 20.57
20220208T080101_20220208T082541_T35HKC 2022-02-08 08:29 100.00 100.00 PASSED PASSED 64.36 34.34 180.77 3.07 22.98
20220218T080001_20220218T082230_T35HKC 2022-02-18 08:29 100.00 99.97 PASSED PASSED 59.90 36.50 181.99 3.06 22.38
20220223T075919_20220223T082859_T35HKC 2022-02-23 08:29 100.00 100.00 PASSED PASSED 57.62 37.67 181.52 3.06 25.82
20220228T075851_20220228T082659_T35HKC 2022-02-28 08:29 100.00 100.00 PASSED PASSED 55.24 38.84 182.47 3.06 28.50
20220305T075809_20220305T082125_T35HKC 2022-03-05 08:29 100.00 99.64 PASSED PASSED 52.93 40.09 181.19 3.06 26.51
20220310T075741_20220310T082729_T35HKC 2022-03-10 08:29 100.00 97.49 PASSED PASSED 50.60 41.35 182.55 3.06 13.33
20220315T075659_20220315T082125_T35HKC 2022-03-15 08:29 100.00 96.10 PASSED PASSED 48.38 42.68 182.00 3.06 15.17
20220330T075611_20220330T082727_T35HKC 2022-03-30 08:29 100.00 100.00 PASSED PASSED 42.13 46.78 181.65 3.07 21.63
Compositing¶
Collections can be composited using composite(). By default, cloud is masked in the component images before compositing. E.g. to form a cloud-free median composite:
# create and filter a collection
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
filt_coll = coll.gd.filter('2021-10-01', '2022-04-01', region=region)
# composite
comp_im = filt_coll.gd.composite(method='median')
CompositeMethod documents supported values for the method parameter. The mosaic, q_mosaic, and medoid methods prioritise images in their sort order i.e. when more than one image pixel qualifies for selection, they select the first one. Images can be sorted by closeness to the date parameter, or by cloud-free portion of the region parameter. If neither date or region are supplied, images are sorted by capture date.
Exporting¶
Preparation¶
Images are exported with the projection and bounds given by their crs, transform and shape properties; and with data type given by their dtype property:
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
print(im.gd.crs)
print(im.gd.transform)
print(im.gd.shape)
print(im.gd.dtype)
EPSG:32735
(10, 0, 199980, 0, -10, 6300040)
(10980, 10980)
uint32
Collections are exported with the projection, bounds and data type given by the first collection image.
Both the image and collection accessors have a prepareForExport() method with the same parameters. This can be called before exporting to change the projection, bounds and data type:
Note
This is required for:
images without a fixed projection (e.g. composites)
collections whose images don’t have a fixed projection, or don’t share the same projection, bounds and data type
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(
crs='EPSG:3857', region=region, scale=30, dtype='uint16'
)
Projection and bounds can be defined with the crs, region and scale / shape; or crs, crs_transform and shape parameters. Other parameters alter resampling, selected bands and scale / offset - see the ee.Image.gd.prepareForExport() or ee.ImageCollection.gd.prepareForExport() docs for details.
GeoTIFF¶
Image¶
ee.Image.gd.toGeoTIFF() exports an image to a GeoTIFF file:
# create and prepare an image
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(region=region, scale=30, dtype='uint16')
# export
prep_im.gd.toGeoTIFF('s2.tif')
import rasterio as rio
with rio.open('s2.tif') as ds:
# default namespace tags
print(ds.tags())
# band 1 tags
print(ds.tags(bidx=1))
{'AOT_RETRIEVAL_ACCURACY': '0', 'CLOUDY_PIXEL_PERCENTAGE': '7.464998', 'CLOUD_COVERAGE_ASSESSMENT': '7.464998', 'CLOUD_SHADOW_PERCENTAGE': '0.476946', 'DARK_FEATURES_PERCENTAGE': '2.535131', 'DATASTRIP_ID': 'S2A_OPER_MSI_L2A_DS_VGS2_20211220T100229_S20211220T082827_N03.01', 'DATATAKE_IDENTIFIER': 'GS2A_20211220T080341_033923_N03.01', 'DATATAKE_TYPE': 'INS-NOBS', 'DEGRADED_MSI_DATA_PERCENTAGE': '0', 'FORMAT_CORRECTNESS': 'PASSED', 'GENERAL_QUALITY': 'PASSED', 'GENERATION_TIME': '1639994549000', 'GEOMETRIC_QUALITY': 'PASSED', 'GRANULE_ID': 'L2A_T35HKC_A033923_20211220T082827', 'HIGH_PROBA_CLOUDS_PERCENTAGE': '4.461143', 'LICENSE': 'https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR_HARMONIZED#terms-of-use', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B1': '183.002238919314', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B10': '183.209778422443', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B11': '182.210025828693', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B12': '181.730903604473', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B2': '187.678466476237', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B3': '185.366615351959', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B4': '184.132241490031', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B5': '183.682922936926', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B6': '183.3563508309', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B7': '183.109691838117', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B8': '186.357005411029', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B8A': '182.941314717219', 'MEAN_INCIDENCE_AZIMUTH_ANGLE_B9': '182.928253148734', 'MEAN_INCIDENCE_ZENITH_ANGLE_B1': '3.06615278603653', 'MEAN_INCIDENCE_ZENITH_ANGLE_B10': '2.61190347479859', 'MEAN_INCIDENCE_ZENITH_ANGLE_B11': '2.79184501565815', 'MEAN_INCIDENCE_ZENITH_ANGLE_B12': '2.99404018653664', 'MEAN_INCIDENCE_ZENITH_ANGLE_B2': '2.4284683919248', 'MEAN_INCIDENCE_ZENITH_ANGLE_B3': '2.53509223726786', 'MEAN_INCIDENCE_ZENITH_ANGLE_B4': '2.6563004578057', 'MEAN_INCIDENCE_ZENITH_ANGLE_B5': '2.72988386474952', 'MEAN_INCIDENCE_ZENITH_ANGLE_B6': '2.80815177009197', 'MEAN_INCIDENCE_ZENITH_ANGLE_B7': '2.89096735679217', 'MEAN_INCIDENCE_ZENITH_ANGLE_B8': '2.47809796571547', 'MEAN_INCIDENCE_ZENITH_ANGLE_B8A': '2.97743523242124', 'MEAN_INCIDENCE_ZENITH_ANGLE_B9': '3.16193398356126', 'MEAN_SOLAR_AZIMUTH_ANGLE': '73.8073718283396', 'MEAN_SOLAR_ZENITH_ANGLE': '26.2914700823507', 'MEDIUM_PROBA_CLOUDS_PERCENTAGE': '2.397248', 'MGRS_TILE': '35HKC', 'NODATA_PIXEL_PERCENTAGE': '0', 'NOT_VEGETATED_PERCENTAGE': '39.013728', 'PROCESSING_BASELINE': '03.01', 'PRODUCT_ID': 'S2A_MSIL2A_20211220T080341_N0301_R035_T35HKC_20211220T100229', 'RADIATIVE_TRANSFER_ACCURACY': '0', 'RADIOMETRIC_QUALITY': 'PASSED', 'REFLECTANCE_CONVERSION_CORRECTION': '1.0326712354468', 'SATURATED_DEFECTIVE_PIXEL_PERCENTAGE': '0', 'SENSING_ORBIT_DIRECTION': 'DESCENDING', 'SENSING_ORBIT_NUMBER': '35', 'SENSOR_QUALITY': 'PASSED', 'SNOW_ICE_PERCENTAGE': '0.00505', 'SOLAR_IRRADIANCE_B1': '1884.69', 'SOLAR_IRRADIANCE_B10': '367.15', 'SOLAR_IRRADIANCE_B11': '245.59', 'SOLAR_IRRADIANCE_B12': '85.25', 'SOLAR_IRRADIANCE_B2': '1959.66', 'SOLAR_IRRADIANCE_B3': '1823.24', 'SOLAR_IRRADIANCE_B4': '1512.06', 'SOLAR_IRRADIANCE_B5': '1424.64', 'SOLAR_IRRADIANCE_B6': '1287.61', 'SOLAR_IRRADIANCE_B7': '1162.08', 'SOLAR_IRRADIANCE_B8': '1041.63', 'SOLAR_IRRADIANCE_B8A': '955.32', 'SOLAR_IRRADIANCE_B9': '812.92', 'SPACECRAFT_NAME': 'Sentinel-2A', 'system-asset_size': '1735937022', 'system-footprint': "{'geodesic': False, 'crs': {'type': 'name', 'properties': {'name': 'EPSG:32735'}}, 'type': 'Polygon', 'coordinates': [[[254220, 6262390], [263820, 6262390], [263820, 6273760], [254220, 6273760], [254220, 6262390]]]}", 'system-index': '20211220T080341_20211220T082827_T35HKC', 'system-time_end': '1639988982907', 'system-time_start': '1639988982907', 'THIN_CIRRUS_PERCENTAGE': '0.606607', 'UNCLASSIFIED_PERCENTAGE': '1.75378', 'VEGETATION_PERCENTAGE': '21.298875', 'WATER_PERCENTAGE': '27.451491', 'WATER_VAPOUR_RETRIEVAL_ACCURACY': '0', 'AREA_OR_POINT': 'Area'}
{'center_wavelength': '0.4439', 'description': 'Aerosols', 'gee-scale': '0.0001', 'gee-wavelength': '443.9nm (S2A) / 442.3nm (S2B)', 'gsd': '60', 'name': 'B1'}
Collection¶
ee.ImageCollection.gd.toGeoTIFF() exports a collection to GeoTIFF files. The split parameter controls whether exported files correspond to collection bands or images:
from pathlib import Path
# create and prepare a collection (with two images and three bands)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = coll.filterBounds(region).limit(2)
prep_coll = coll.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# create export directory
dirname = Path('s2')
dirname.mkdir()
# export (one file for each collection band)
prep_coll.gd.toGeoTIFF(dirname, split='bands')
# display exported files
[fp.name for fp in dirname.glob('*.tif')]
['B3.tif', 'B2.tif', 'B4.tif']
When split is images, image properties are written to the GeoTIFF default namespace tags, and bandProps are written to the band tags (see the image example).
Nodata¶
By default, GeoTIFF file nodata tags are set to the nodata value of their corresponding images. Both ee.Image.gd.toGeoTIFF() and ee.ImageCollection.gd.toGeoTIFF() have a nodata parameter that allows this to be changed. E.g.:
# set masked pixels to a new nodata value
nodata = 65535
prep_im = prep_im.unmask(nodata)
# export, setting nodata to the new value
prep_im.gd.toGeoTIFF('s2_nodata.tif', nodata=nodata)
# display GeoTIFF nodata
with rio.open('s2_nodata.tif') as ds:
print(ds.nodata)
65535.0
Paths and URIs¶
The file argument in ee.Image.gd.toGeoTIFF() and dirname argument in ee.ImageCollection.gd.toGeoTIFF() can be local paths or remote URIs. See the related note in the command line section for more information.
NumPy¶
Image¶
ee.Image.gd.toNumPy() exports an image to a NumPy ndarray:
# create and prepare an image (with 3 bands)
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export (3D array with bands along the third dimension)
array = prep_im.gd.toNumPy()
# display array format
print(type(array))
print(array.shape)
print(array.dtype)
<class 'numpy.ndarray'>
(379, 320, 3)
uint16
Collection¶
ee.ImageCollection.gd.toNumPy() exports a collection to a NumPy ndarray. The split parameter controls the layout of collection bands and images in the exported array:
# create and prepare a collection (with two images and three bands)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = coll.filterBounds(region).limit(2)
prep_coll = coll.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export (4D array with bands along the third, and images along the fourth dimension)
array = prep_coll.gd.toNumPy(split='bands')
# display array format
print(type(array))
print(array.shape)
print(array.dtype)
<class 'numpy.ndarray'>
(379, 320, 3, 2)
uint16
Masking and data type¶
Both ee.Image.gd.toNumPy() and ee.ImageCollection.gd.toNumPy() have masked and structured parameters. The masked parameter controls whether the exported array has masked pixels set to nodata, or is a MaskedArray. The structured parameter controls whether the exported array has a numerical or structured data type. E.g.:
# export (2D masked array with a structured dtype representing the bands)
array = prep_im.gd.toNumPy(masked=True, structured=True)
# display array format
print(type(array))
print(array.shape)
print(array.dtype)
<class 'numpy.ma.MaskedArray'>
(379, 320)
[('B4', '<u2'), ('B3', '<u2'), ('B2', '<u2')]
Xarray¶
Image¶
ee.Image.gd.toXarray() exports an image to a Xarray DataArray:
# create and prepare image
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export (3D DataArray)
da = prep_im.gd.toXarray()
da
<xarray.DataArray (y: 379, x: 320, band: 3)> Size: 728kB 427 450 343 991 867 532 913 1058 730 ... 964 882 650 998 918 756 1033 996 797 Coordinates: * y (y) float64 3kB 6.274e+06 6.274e+06 ... 6.262e+06 6.262e+06 * x (x) float64 3kB 2.542e+05 2.543e+05 ... 2.638e+05 2.638e+05 * band (band) <U2 24B 'B4' 'B3' 'B2' Attributes: (7)
Collection¶
ee.ImageCollection.gd.toXarray() exports a collection to a Xarray Dataset. The split parameter controls whether dataset variables correspond to collection bands or images:
# create and prepare a collection (with two images and three bands)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = coll.filterBounds(region).limit(2)
prep_coll = coll.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export (Dataset with bands as variables)
ds = prep_coll.gd.toXarray(split='bands')
ds
<xarray.Dataset> Size: 1MB
Dimensions: (y: 379, x: 320, time: 2)
Coordinates:
* y (y) float64 3kB 6.274e+06 6.274e+06 ... 6.262e+06 6.262e+06
* x (x) float64 3kB 2.542e+05 2.543e+05 ... 2.638e+05 2.638e+05
* time (time) datetime64[ms] 16B 2015-11-22T08:29:39.439000 2016-02-10T...
Data variables:
B4 (y, x, time) uint16 485kB 390 352 1061 767 435 ... 469 454 472 444
B3 (y, x, time) uint16 485kB 540 477 919 739 573 ... 441 414 430 409
B2 (y, x, time) uint16 485kB 257 234 606 442 285 ... 283 267 255 229
Attributes: (6)Masking¶
Both ee.Image.gd.toXarray() and ee.ImageCollection.gd.toXarray() have a masked parameter that controls whether exported masked pixels are set to nodata, or to NaN. If they are set to NaN, the export data type will be converted to a floating point type able to represent the data:
# create and prepare a cloud masked image
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
im = im.gd.addMaskBands().gd.maskClouds()
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export, setting masked pixels to NaN
da = prep_im.gd.toXarray(masked=True)
# check for NaN pixels and floating point data type
print(da.isnull().any())
print(da.dtype)
<xarray.DataArray ()> Size: 1B
True
Attributes: (7)
float32
See the Xarray documentation on missing values for background.
Attributes¶
DataArray / Dataset attributes include crs, transform and nodata values for compatibility with rioxarray, as well as ee and stac JSON strings of the Earth Engine property and STAC dictionaries.
Google cloud¶
Image¶
ee.Image.gd.toGoogleCloud() exports an image to Google Drive, Earth Engine asset or Google Cloud Storage:
# create and prepare image
im = ee.Image('COPERNICUS/S2_SR_HARMONIZED/20211220T080341_20211220T082827_T35HKC')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
prep_im = im.gd.prepareForExport(region=region, scale=30, dtype='uint16')
# export to Earth Engine asset 's2' in the 'geedim' project, waiting for completion
_ = prep_im.gd.toGoogleCloud('s2', type='asset', folder='geedim', wait=True)
# display asset image info
ee.Image('projects/geedim/assets/s2').getInfo()
{'type': 'Image', 'bands': [{'id': 'B1', 'data_type': {'type': 'PixelType', ...
Collection¶
ee.ImageCollection.gd.toGoogleCloud() exports a collection to Google Drive, Earth Engine asset or Google Cloud Storage. The split parameter controls whether exported files / assets correspond to collection bands or images:
# create and prepare a collection (with two images and three bands)
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = coll.filterBounds(region).limit(2)
prep_coll = coll.gd.prepareForExport(
region=region, scale=30, dtype='uint16', bands=['B4', 'B3', 'B2']
)
# export to Earth Engine assets in the 'geedim' project, waiting for completion
# (one asset for each collection band)
_ = prep_coll.gd.toGoogleCloud(type='asset', folder='geedim', wait=True, split='bands')
# display the info of the first asset image
ee.Image('projects/geedim/assets/B4').getInfo()
{'type': 'Image', 'bands': [{'id': 'B_20180510T075611_20180510T082300_T35HKC', ...
Additional arguments¶
Depending on the type parameter, toGoogleCloud() calls one of the Export.image.toDrive(), Export.image.toAsset() or Export.image.toCloudStorage() Earth Engine functions to perform the export. ee.Image.gd.toGoogleCloud() and ee.ImageCollection.gd.toGoogleCloud() allow additional keyword arguments to be passed to the type relevant Earth Engine function. See the Export.image.toDrive(), Export.image.toAsset() or Export.image.toCloudStorage() docs for supported parameters. E.g.
# export to Google Drive using the TFRecord format
_ = prep_im.gd.toGoogleCloud(
's2',
type='drive',
folder='geedim',
fileFormat='TFRecord',
formatOptions={'patchDimensions': [256, 256], 'compressed': True},
)
Tiling¶
The toGeoTIFF(), toNumPy() and toXarray() methods divide images into tiles for export. Tiles are downloaded and decompressed concurrently, then reassembled into the target export format. Tile size can be controlled with the max_tile_size, max_tile_dim and max_tile_bands parameters. Download concurrency can be controlled with the max_requests, and decompress concurrency with the max_cpus parameter. Each parameter has an upper limit - see the toGeoTIFF(), toNumPy() or toXarray() reference docs for details. For most uses, the tiling parameters can be left on their default values.
User memory limit error¶
Exporting computed images with toGeoTIFF(), toNumPy() or toXarray() could raise a 'User memory limit exceeded' error in some unusual cases. Earth Engine raises this error if a computation exceeds the limit on user memory. E.g.:
# create a 2 year cloud-free median composite
coll = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
region = ee.Geometry.Rectangle(24.35, -33.75, 24.45, -33.65)
coll = coll.gd.filter('2021-01-01', '2023-01-01', region=region)
comp_im = coll.gd.composite(method='median')
# prepare the composite for export
prep_im = comp_im.gd.prepareForExport(
crs='EPSG:3857', region=region, scale=10, dtype='uint16'
)
# attempt export to NumPy array
array = prep_im.gd.toNumPy()
---------------------------------------------------------------------------
ClientResponseError Traceback (most recent call last)
Cell In[23], line 13
9 crs='EPSG:3857', region=region, scale=10, dtype='uint16'
10 )
11
12 # attempt export to NumPy array
---> 13 array = prep_im.gd.toNumPy()
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/image.py:1165, in ImageAccessor.toNumPy(self, masked, structured, max_tile_size, max_tile_dim, max_tile_bands, max_requests, max_cpus)
1155 array[tile.slices.row, tile.slices.col, tile.slices.band] = tile_array
1157 with Tiler(
1158 self,
1159 max_tile_size=max_tile_size,
(...) 1163 max_cpus=max_cpus,
1164 ) as tiler:
-> 1165 tiler.map_tiles(write_tile, masked=masked)
1167 if structured:
1168 dtype = np.dtype(
1169 dict(names=self.bandNames, formats=[self.dtype] * len(self.bandNames))
1170 )
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/tile.py:426, in Tiler.map_tiles(self, func, masked)
424 # download tiles using persistent session
425 runner = utils.AsyncRunner()
--> 426 runner.run(_map_tiles(runner.session))
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/utils.py:389, in AsyncRunner.run(self, coro, **kwargs)
383 if loop and loop.is_running():
384 # run in a separate thread if there is an existing loop (e.g. we are in a
385 # jupyter notebook)
386 self._executor = self._executor or ThreadPoolExecutor(max_workers=1)
387 return self._executor.submit(
388 lambda: self._runner.run(coro, **kwargs)
--> 389 ).result()
390 else:
391 # run in this thread if there is no existing loop
392 return self._runner.run(coro, **kwargs)
File ~/.asdf/installs/python/3.14.0/lib/python3.14/concurrent/futures/_base.py:450, in Future.result(self, timeout)
448 raise CancelledError()
449 elif self._state == FINISHED:
--> 450 return self.__get_result()
451 else:
452 raise TimeoutError()
File ~/.asdf/installs/python/3.14.0/lib/python3.14/concurrent/futures/_base.py:395, in Future.__get_result(self)
393 if self._exception is not None:
394 try:
--> 395 raise self._exception
396 finally:
397 # Break a reference cycle with the exception in self._exception
398 self = None
File ~/.asdf/installs/python/3.14.0/lib/python3.14/concurrent/futures/thread.py:86, in _WorkItem.run(self, ctx)
83 return
85 try:
---> 86 result = ctx.run(self.task)
87 except BaseException as exc:
88 self.future.set_exception(exc)
File ~/.asdf/installs/python/3.14.0/lib/python3.14/concurrent/futures/thread.py:73, in WorkerContext.run(self, task)
71 def run(self, task):
72 fn, args, kwargs = task
---> 73 return fn(*args, **kwargs)
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/utils.py:388, in AsyncRunner.run.<locals>.<lambda>()
383 if loop and loop.is_running():
384 # run in a separate thread if there is an existing loop (e.g. we are in a
385 # jupyter notebook)
386 self._executor = self._executor or ThreadPoolExecutor(max_workers=1)
387 return self._executor.submit(
--> 388 lambda: self._runner.run(coro, **kwargs)
389 ).result()
390 else:
391 # run in this thread if there is no existing loop
392 return self._runner.run(coro, **kwargs)
File ~/.asdf/installs/python/3.14.0/lib/python3.14/asyncio/runners.py:127, in Runner.run(self, coro, context)
125 self._interrupt_count = 0
126 try:
--> 127 return self._loop.run_until_complete(task)
128 except exceptions.CancelledError:
129 if self._interrupt_count > 0:
File ~/.asdf/installs/python/3.14.0/lib/python3.14/asyncio/base_events.py:719, in BaseEventLoop.run_until_complete(self, future)
716 if not future.done():
717 raise RuntimeError('Event loop stopped before Future completed.')
--> 719 return future.result()
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/tile.py:414, in Tiler.map_tiles.<locals>._map_tiles(session)
410 with utils.auto_leave_tqdm(
411 asyncio.as_completed(tasks), total=len(tasks), **tqdm_kwargs
412 ) as bar:
413 for task in bar:
--> 414 await task
416 except:
417 # cancel and await any incomplete tasks
418 logger.debug('Cleaning up export tasks...')
File ~/.asdf/installs/python/3.14.0/lib/python3.14/asyncio/tasks.py:618, in _AsCompletedIterator._wait_for_one(self, resolve)
615 if f is None:
616 # Dummy value from _handle_timeout().
617 raise exceptions.TimeoutError
--> 618 return f.result() if resolve else f
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/tile.py:356, in Tiler._map_tile(self, func, tile, masked, session, max_retries, backoff_factor)
354 url = await loop.run_in_executor(self._executor, get_tile_url)
355 logger.debug(f'Downloading {tile!r} from {url}.')
--> 356 mem_file = await download_url(url)
358 # enter memory file context (it must be closed when done)
359 with mem_file:
360 # limit concurrent tile reads to leave CPU capacity for the event
361 # loop
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/geedim/tile.py:337, in Tiler._map_tile.<locals>.download_url(url)
335 except aiohttp.ClientError:
336 pass
--> 337 response.raise_for_status()
339 async for data, _ in response.content.iter_chunks():
340 mem_file.write(data)
File ~/checkouts/readthedocs.org/user_builds/geedim/envs/latest/lib/python3.14/site-packages/aiohttp/client_reqrep.py:655, in ClientResponse.raise_for_status(self)
652 if not self._in_context:
653 self.release()
--> 655 raise ClientResponseError(
656 self.request_info,
657 self.history,
658 status=self.status,
659 message=self.reason,
660 headers=self.headers,
661 )
ClientResponseError: 400, message='User memory limit exceeded.', url='https://earthengine-highvolume.googleapis.com/v1/projects/geedim/thumbnails/313e551072ecb05467cfc9af33db22ea-184799ab3aa113cb044b4fbf6885bed6:getPixels'
toGoogleCloud() is not subject to the limit and using it for export is recommended in this situation. Images can first be exported to Earth Engine asset with toGoogleCloud(), and then the computed assets exported to a target format with one of toGeoTIFF(), toNumPy() or toXarray(). E.g.:
# export composite to Earth Engine asset 's2-comp' in the 'geedim' project
_ = prep_im.gd.toGoogleCloud('s2-comp', type='asset', folder='geedim', wait=True)
# export the asset to a NumPy array
array = ee.Image('projects/geedim/assets/s2-comp').gd.toNumPy()