Processing on AWS

Just did a small example for stacking Sentinel-1 images, but something is still wrong!

The method is mostly what is presented in this repository’s example:

def stack_images(images_path, aoi):
    """
    :param images_path:
    :return:
    """
    absolute_path = os.path.abspath(images_path)
    output_mosaic_path = absolute_path.replace('filter', 'stack')
    aois_path = settings.DATASET_PATH + 'radar/aoi/shp/'
    output_shp_file = aois_path + aoi + ".shp"

    srcfiles = finder(images_path, ['^S1[AB]'], regex=True, recursive=False)
    groups = groupbyTime(srcfiles, seconds, 30)

    try:
        os.makedirs(output_mosaic_path)
    except FileExistsError:
        pass

    stack(srcfiles=groups, dstfile=output_mosaic_path + "/" + 'stack.tiff', resampling='bilinear',
          targetres=[20, 20], srcnodata=0, dstnodata=-99,
          shapefile=output_shp_file, sortfun=seconds, separate=False, overwrite=True)

Just to make clear, the srcfiles variable brings all original sentinel images:

['.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093134_20180525T093159_022059_026254_0ED2_iw-vh.tiff', '/.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093159_20180525T093224_022059_026254_DE25_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180525T093224_20180525T093249_022059_026254_28B8_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T093955_20180530T094020_022132_0264A6_04BA_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T094020_20180530T094045_022132_0264A6_A27A_iw-vh.tiff', '.../iw-vh/S1A_IW_GRDH_1SDV_20180530T094045_20180530T094110_022132_0264A6_D444_iw-vh.tiff']

but still getting this error:

Traceback (most recent call last):
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/auxil.py", line 63, in crsConvert
    srs.SetFromUserInput(crsIn)
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/osgeo/osr.py", line 1068, in SetFromUserInput
    return _osr.SpatialReference_SetFromUserInput(self, *args)
RuntimeError: OGR Error: Corrupt data

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "aws_cli.py", line 114, in <module>
    main(args.ranges)
  File "aws_cli.py", line 70, in main
    dip.stack_images(path_to_pol, aoi)
  File "/home/rodolfo/Bit/lotte/aws-imagery/aws_process.py", line 82, in stack_images
    shapefile=output_shp_file, sortfun=seconds, separate=False, overwrite=True)
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/raster.py", line 1231, in stack
    group = [x for x in group if intersect(shp, Raster(x).bbox())]
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/raster.py", line 1231, in <listcomp>
    group = [x for x in group if intersect(shp, Raster(x).bbox())]
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/raster.py", line 426, in bbox
    return bbox(coordinates=extent, crs=crs)
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/vector.py", line 725, in bbox
    srs = crsConvert(crs, 'osr')
  File "/home/rodolfo/Bit/lotte/aws-imagery/.venv/lib/python3.7/site-packages/spatialist/auxil.py", line 66, in crsConvert
    '  was: "{}" of type {}'.format(crsIn, type(crsIn).__name__))
TypeError: crsIn not recognized; must be of type WKT, PROJ4 or EPSG
  was: "" of type str

Seems to be some projection missing data! But I’m using original image and check with another softwares, which gave me the correct metadata!

[johntruckenbrodt]

A SAR scene usually consists of one or more image files and in most cases several text files containing metadata. If you want to process a SAR scene with the geocode function, you need to provide the whole scene. In the case of Sentinel-1 this is either the .zip archive as downloaded from e.g. the Copernicus Open Access Hub or the folder ending with .SAFE.
So instead of the direct path to the individual .tiff files, provide a list of paths to the .zip or .SAFE SAR scene locations. Beware however, that a list of SAR scenes is “treated as consecutive acquisitions, which will be mosaicked with SNAP’s SliceAssembly operator” as described in the geocode documentation. If your list of scenes spans a larger time frame, it is better to first group them using function groupbyTime and then pass each group to geocode:

from pyroSAR.snap import geocode
from pyroSAR.ancillary import groupbyTime, seconds
from spatialist.ancillary import finder
list_images = finder('/location', ['*.SAFE'], foldermode=2)
list_images = groupbyTime(images=list_images, function=seconds, time=30)
for group in list_images:
    geocode(group, ...)

The function spatialist.finder is just for demonstration, but seems not to work for AWS S3 yet as discussed in issue johntruckenbrodt/spatialist#10.

[johntruckenbrodt]

Sure, you’re welcome. I am closing this for now but feel free to reopen it if you encounter something similar.