dep_tools.aws module

Utility / helper functions for using Amazon S3 Storage.

dep_tools.aws.object_exists(bucket, key, client=None)[source]

Check if a given object exists in a bucket.

Parameters:

  • bucket (str) – The name of the bucket.

  • key (str) – The object key.

  • client (Optional[BaseClient]) – An optional client. If none is specified the default client is used.

Return type:

bool

Returns:

True if the object is in the bucket, otherwise False.

dep_tools.aws.s3_dump(data, bucket, key, client, **kwargs)[source]

Write data to an s3 bucket.

This is a wrapper around boto3.client.put_object().

Parameters:

  • data (Union[bytes, str, IO]) – The data to write.

  • bucket (str) – The name of the bucket.

  • key (str) – The key in the bucket where the data should go.

  • client (BaseClient) – A client.

  • **kwargs – Additional arguments to boto3.client.put_object()

Return type:

bool

Returns:

True if the operation was successful, otherwise False.

dep_tools.aws.write_to_s3(d, path, bucket, overwrite=True, use_odc_writer=True, client=None, s3_dump_kwargs={}, **kwargs)[source]

Writes a given object to s3.

This is a specialized version of s3_dump(). If the object is a GeoDataFrame, geopandas.to_file() is used. If an xarray DataArray or Dataset, odc.geo.xr.to_cog() is used if use_odc_writer is True, otherwise rioxarray.to.raster() is used. If a pystac.Item, d is first dumped to json.

Parameters:

  • d (Union[DataArray, Dataset, GeoDataFrame, Item, str]) – The data or object to write.

  • path (Union[str, Path]) – The path (key) to write the object.

  • bucket (str) – The bucket where the object should go.

  • overwrite (bool) – Whether existing objects with the same key should be overwritten.

  • use_odc_writer (bool) – Whether odc.geo.xr.to_cog() should be used to write the object. If False rioxarray.to_raster() is used. Only matters for xarray objects (DataArrays or Datasets).

  • client (Optional[BaseClient]) – The s3 client. If not set the default client is used.

  • () (s3_dump_kwargs) – Additional arguments to s3_dump().

  • **kwargs – Additional arguments to the writing function, if the object is a geopandas.GeoDataFrame or xarray.DataArray / xarray.Dataset.

Raises:

ValueError – If d is not of the allowed types.

dep_tools.aws.write_stac_s3(item, stac_path, bucket, **kwargs)[source]

Writes a STAC item to s3.

Parameters:

  • item (Item) – A STAC item.

  • stac_path (str) – The path where the item should go.

  • bucket (str) – The bucket where the item should go.

  • **kwargs – Additional arguments to write_to_s3().

Return type:

str

Returns:

The stac_path.

dep_tools.aws.get_s3_bucket_region(bucket_name, client=None)[source]

Return the AWS region for a given bucket.

Parameters:

bucket_name (str) – The name of the bucket

Return type:

str

Returns:

The name of the region.