"Copying directories not supported" on AWS

Ran a workflow using V40 Cromwell on AWSBATCH that had as outputs (one outputfile.vcf for each of the shards in the workflow):

output {
    Array[File] outputs = task.outputvcf
}

I used the following workflow options:

{
    "workflow_failure_mode": "NoNewCalls",
    "default_runtime_attributes": {
        "maxRetries": 1
    },
    "final_workflow_outputs_dir": "s3://bucket/Cromwell/results",
    "use_relative_output_paths": "false",
    "final_workflow_log_dir": "s3://bucket/Cromwell/workflowLogs",
    "final_call_logs_dir": "s3://bucket/Cromwell/workflowLogs"
}

All calls of the workflow completed successfully but the workflow itself failed.

Error Message I got: "copying directories is not yet supported: s3://s3.amazonaws.com/bucket/Cromwell/results/workflowName/1ec38d0b-afc4-4cd5-90f1-f015395d6e36/call-task/shard-0/outputfile.vcf"

Oddly enough, the correct prefixes for the output files were created in the correct S3 bucket, they just don’t have an object there, and via the CLI they appear as directories. ???

For the logs, a prefix was made that is empty, and the log file was written successfully to one level higher than the prefix it is supposed to be in. So instead of:

s3://bucket/Cromwell/workflowLogs/workflowName/<workflowid>.log

there is:

s3://bucket/Cromwell/workflowLogs/workflowName/ (empty prefix)
s3://bucket/Cromwell/workflowLogs/<workflowid>.log (successfully written file)

Thoughts?

[svitkovsergey]

@wleepang @geoffjentry Any updates about PR #5110 ? 😄

[robthompsonweb]

@TimurKustov correct (apart from the trailing slash on output.txt… not sure if that is a typo but that will break it on s3. it would copy output.txt to s3://3-bucket/WGS_BAM_to_GVCF/workflow/call-MergeGVCFs/output.txt. ie aws s3 cp localpath/WGS_BAM_to_GVCF/workflow/call-MergeGVCFs/output.txt s3://3-bucket/WGS_BAM_to_GVCF/workflow/call-MergeGVCFs/output.txt

I would expect it to copy every file from the output into the path specified by final_workflow_outputs_dir. if use_relative_output_paths is false it would be: s3://3-bucket/output.txt