CWL workflows cannot read from S3

As noted by @boyangzhao in https://github.com/DataBiosphere/toil/issues/4091#issuecomment-1122732414, toil-cwl-runner does not seem to be able to read a CWL File object with a location set to an S3 URL:

txt_file:
  class: File
  location: s3://test/test.cwl

We should fix this, and add a test so this can’t break again.

This might be a problem for the upcoming AGC release.

┆Issue is synchronized with this Jira Story ┆friendlyId: TOIL-1165

[boyangzhao]

thanks! It looks like it fixed now, and confirm it’s working on my test examples.

[adamnovak]

OK, I’ve managed to reproduce the problem you are seeing. It seems to only happen when you run a single CommandLineTool as your main CWL document, and wrapping it in a Workflow makes the problem go away.

I’ve opened #4125 to track this.