relative imports from local-submitted CWL seems to fail

Running a CWL workflow with local relative imports seems to fail. This is possibly related to #4308 but unsure.

Running with cromwell 36.

Process looks like the following

$ git clone https://github.com/dockstore-testing/dockstore-workflow-md5sum-unified.git
$ cd dockstore-workflow-md5sum-unified
$ cwltool checker_workflow_wrapping_workflow.cwl md5sum.json
/usr/local/bin/cwltool 1.0.20180403145700
<snip>
Final process status is success
$ wget https://github.com/broadinstitute/cromwell/releases/download/36/cromwell-36.jar
$ java -jar cromwell-36.jar  run https://raw.githubusercontent.com/dockstore-testing/dockstore-workflow-md5sum-unified/develop/checker_workflow_wrapping_workflow.cwl --inputs md5sum.json
<snip>
[2018-11-07 14:34:25,13] [info] Pre-Processing /tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl
[2018-11-07 14:34:34,94] [error] WorkflowManagerActor Workflow a3cb6a14-3672-4132-8a24-2e0a4e66ff96 failed (during MaterializingWorkflowDescriptorState): cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anon$1: Workflow input processing failed:
running cwltool on file /tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl failed with Traceback (most recent call last):
  File "/home/dyuen/dockstore_tools/dockstore-workflow-md5sum-unified/cromwell-36.jar/Lib/heterodon/__init__.py", line 24, in apply
  File "<string>", line 1, in <module>
  File "<string>", line 12, in cwltool_salad
  File "/home/dyuen/dockstore_tools/dockstore-workflow-md5sum-unified/cromwell-36.jar/Lib/cwltool/load_tool.py", line 279, in validate_document
  File "/home/dyuen/dockstore_tools/dockstore-workflow-md5sum-unified/cromwell-36.jar/Lib/schema_salad/ref_resolver.py", line 915, in resolve_all
  File "/home/dyuen/dockstore_tools/dockstore-workflow-md5sum-unified/cromwell-36.jar/Lib/schema_salad/ref_resolver.py", line 1087, in validate_links
schema_salad.validate.ValidationException: ../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl:24:1: checking field steps
../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl:30:3:   checking object ../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl#checker
../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl:31:5:     Field run contains undefined reference to file:///tmp/cwl_temp_dir_7264114231127246601/checker/md5sum_checker.cwl
../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl:25:3:   checking object ../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl#md5sum
../../../../tmp/cwl_temp_dir_7264114231127246601/cwl_temp_file_a3cb6a14-3672-4132-8a24-2e0a4e66ff96.cwl:26:5:     Field run contains undefined reference to file:///tmp/cwl_temp_dir_7264114231127246601/md5sum/md5sum.cwl
<snip>
$ java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Inspecting the directory /tmp/cwl_temp_dir_7264114231127246601 the relative imports are indeed not present (or anything else really)

[cjllanwarne]

Hi @denis-yuen - I’ll have a look at this after completing my fix for “relative imports from remote URLs” - hopefully this will either be taken care of for free or a very similar fix

[multimeric]

Did you try using --imports .? e.g.

java -jar cromwell-36.jar run checker_workflow_wrapping_workflow.cwl  --inputs md5sum.json --imports .

It looks like it’s moving your CWL file into a temporary directory for running, but missing out on the other files it imports. It seems like --imports would tell Cromwell about the imports