"No coercion defined" error in optional output in CWL with wildcard glob.

When trying one of our new CWL workflow steps in Cromwell, we’re encountering an unexpected error with an optional output when that output isn’t present:

Bad output 'test.cwl.not_found': No coercion defined from wom value(s) '[]' of type 'Array[Nothing]' to 'class wom.types.WomMaybePopulatedFileType$?'.

This only seems to occur when the glob in the outputBinding contains a *. A small test case (run with /usr/bin/java -jar cromwell-34.jar run -t cwl -i test.yml test.cwl demonstrates this:

test.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: CommandLineTool
requirements:
    - class: ShellCommandRequirement
baseCommand: ['/bin/echo', "this is a"]
arguments: [
    { valueFrom: '>', shellQuote: false },
    'some_output.txt'
]
inputs:
    bonus:
        type: string
        inputBinding:
            position: -1
outputs:
    found:
        type: File
        outputBinding:
            glob: '*output.txt'
    not_found:
        type: File?
        outputBinding:
            glob: '*extra.txt'

test.yaml

bonus: "test"

cwltool handles this case as expected:

$ cwltool test.cwl test.yml
/usr/local/bin/cwltool 1.0.20170822192924
Resolved 'test.cwl' to 'file:///home/tmooney/cromwell_test/glob/test.cwl'
[job test.cwl] /tmp/tmpqeLl9_$ /bin/sh \
    -c \
    '/bin/echo' 'this is a' 'test' > 'some_output.txt'
[job test.cwl] completed success
{
    "found": {
        "checksum": "sha1$6476df3aac780622368173fe6e768a2edc3932c8", 
        "basename": "some_output.txt", 
        "nameext": ".txt", 
        "nameroot": "some_output", 
        "location": "file:///home/tmooney/cromwell_test/glob/some_output.txt", 
        "path": "/home/tmooney/cromwell_test/glob/some_output.txt", 
        "class": "File", 
        "size": 15
    }, 
    "not_found": null
}
Final process status is success

Cromwell fails with this error:

[2018-08-14 16:14:05,89] [info] MaterializeWorkflowDescriptorActor [a3d3e011]: Parsing workflow as CWL v1.0
[2018-08-14 16:14:07,03] [info] MaterializeWorkflowDescriptorActor [a3d3e011]: Call-to-Backend assignments: test.cwl -> Local
[2018-08-14 16:14:10,44] [info] WorkflowExecutionActor-a3d3e011-3a0c-4203-9edb-3d65564a1d1d [a3d3e011]: Starting test.cwl
[2018-08-14 16:14:11,85] [info] BackgroundConfigAsyncJobExecutionActor [a3d3e011test.cwl:NA:1]: '/bin/echo' 'this is a' 'test' > 'some_output.txt'
[2018-08-14 16:14:11,97] [info] BackgroundConfigAsyncJobExecutionActor [a3d3e011test.cwl:NA:1]: executing: /bin/bash /home/tmooney/cromwell_test/glob/cromwell-executions/test.cwl/a3d3e011-3a0c-4203-9edb-3d65564a1d1d/call-test.cwl/execution/script
[2018-08-14 16:14:13,16] [info] BackgroundConfigAsyncJobExecutionActor [a3d3e011test.cwl:NA:1]: job id: 1832
[2018-08-14 16:14:13,17] [info] BackgroundConfigAsyncJobExecutionActor [a3d3e011test.cwl:NA:1]: Status change from - to WaitingForReturnCodeFile
[2018-08-14 16:14:14,45] [info] BackgroundConfigAsyncJobExecutionActor [a3d3e011test.cwl:NA:1]: Status change from WaitingForReturnCodeFile to Done
[2018-08-14 16:14:15,67] [error] WorkflowManagerActor Workflow a3d3e011-3a0c-4203-9edb-3d65564a1d1d failed (during ExecutingWorkflowState): cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs:
Bad output 'test.cwl.not_found': No coercion defined from wom value(s) '[]' of type 'Array[Nothing]' to 'class wom.types.WomMaybePopulatedFileType$?'.
        at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:839)
        at scala.util.Success.$anonfun$map$1(Try.scala:251)
        at scala.util.Success.map(Try.scala:209)
        at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
        at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
        at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[Horneth]

@ruchim I’d expect Cromwell to give you an empty optional file if nothing matched, a defined optional file if only one file matched, and fail if more than one file matched.

[tmooney]

@ruchim Yes! We eventually switched our particular workflow that motivated this issue to avoid a glob altogether. We really do expect at most one matching file, so this was simpler than pulling it out of an array later.