batch.imagecollection.toCloudStorage does not upload all files

Thank you for making this functionality available.

I’m been trying to export some images on my google cloud storage, but am running into trouble. Below is an example of my run when I try to upload 9 images

BUCKET = "detect_proto"
imageExportFormatOptions = {
      'patchDimensions': [75, 75],  
      'maxFileSize': 100000000,    
      'compressed': True
    }
batch.imagecollection.toCloudStorage(newImgCol, bucket=BUCKET, folder="geetools", scale=10, 
                                     fileFormat = 'TFRecord', verbose=True,
                                     formatOptions = imageExportFormatOptions)

This will yield the following output

adding task 0 to list
adding task 1 to list
adding task 2 to list
adding task 3 to list
adding task 4 to list
adding task 5 to list
adding task 6 to list
adding task 7 to list
adding task 8 to list

But when I check my cloud storage, only 1 file has been uploaded

gsutil ls gs://detect_proto/geetools

gs://detect_proto/geetools/0.json
gs://detect_proto/geetools/0.tfrecord.gz

I’m a bit confused why only 1 file has been uploaded, any thoughts on how to investigate this would be greatly appreciated. CC @MarcCoru @fitoprincipe

[csong21]

I updated my code and this seems to work. The only change that enabled it to work is that I moved the imageExportFormatOptions within the for loop.

for x, y in neighbors:

    imgs = ee.ImageCollection('COPERNICUS/S2').filterDate(str(last_month), str(today_date)).filterBounds(ee.Geometry.Point([lon, lat])).map(maskS2clouds)
    # Get the latest image for the bounding area
    lastimg = imgs.limit(1, 'system:time_start', False).first()
    # Preprocess the image the same way as the training data
    lastimg = lastimg.select(bands).multiply(0.0001)
    # Starting from the same export buffer as the training data
    xmin, ymin, xmax, ymax = [x - 0.0025, y - 0.005, x + 0.006, y + 0.0025]
    buffer_rect = ee.Geometry.Rectangle([xmin, ymin, xmax, ymax])
    lastimg_clip = lastimg.clip(buffer_rect)

    imageFilePrefix= 'ProjectID_' + str(project_id) + '_window' + str(w)

    imageExportFormatOptions = {
      'patchDimensions': [75, 75],  
      'maxFileSize': 100000000,    
      'compressed': True
    }


    imageTask = ee.batch.Export.image.toCloudStorage(
          image = lastimg_clip,
          description = 'ExportLastImageClip',
          fileNamePrefix = imageFilePrefix,
          bucket = outputBucket,
          scale = 10,
          fileFormat = 'TFRecord',
          region = buffer_rect,
          formatOptions = imageExportFormatOptions
        )
    imageTask.start()

    print('\n   Exporting window %i to Google Cloud Storage' % w)
          
    while imageTask.active():
      print('\n       Polling for task (id: {}).'.format(imageTask.id))
      time.sleep(5)

    print(imageTask.status()['state'])

    w+=1```

[csong21]

Thanks @MarcCoru for your feedback!