Conditionals doesn't work for unnecessary inputs

Hello,

I’m running a simple wf of 2 steps: move 2 files if forward and reverse reads are presented OR move one file if single_reads is presented (workflows attached).

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2.0-dev2

requirements:
  SubworkflowFeatureRequirement: {}
  MultipleInputFeatureRequirement: {}
  InlineJavascriptRequirement: {}
  StepInputExpressionRequirement: {}
  ScatterFeatureRequirement: {}


inputs:
    forward_reads: File?
    reverse_reads: File?
    single_reads: File?

outputs: []

steps:

  step_paired:
    run: mv.cwl
    when: $(inputs.single == undefined)
    scatter: initial_file
    in:
      single: single_reads
      initial_file:
        - forward_reads
        - reverse_reads
      out_file_name: { default: 'filename_paired' }
    out: [ renamed_file ]

  step_single:
    run: mv.cwl
    when: $(inputs.single != undefined)
    in:
      single: single_reads
      initial_file: single_reads
      out_file_name: { default: 'filename_single' }
    out: [ renamed_file ]

I do cwltool --no-container --enable-dev wf.cwl wf.yml and it works fine. Example for forward+reverse input:

INFO /usr/local/bin/cwltool 3.0.20200530110633
INFO Resolved 'wf.cwl' to 'file:///Users/kates/Desktop/EBI/CWL/pipeline-v5/workflows/test/wf.cwl'
WARNING Workflow checker warning:
wf.cwl:16:5: Source 'single_reads' of type ["null", "File"] may be incompatible
wf.cwl:39:7:   with sink 'initial_file' of type "File"
wf.cwl:14:5: Source 'forward_reads' of type ["null", "File"] may be incompatible
wf.cwl:28:7:   with sink 'initial_file' of type {"type": "array", "items": "File"}
               source has linkMerge method merge_nested
wf.cwl:15:5: Source 'reverse_reads' of type ["null", "File"] may be incompatible
wf.cwl:28:7:   with sink 'initial_file' of type {"type": "array", "items": "File"}
               source has linkMerge method merge_nested
INFO [workflow ] start
INFO [workflow ] starting step step_single
INFO [step step_single] will be skipped
INFO [step step_single] completed skipped
INFO [workflow ] starting step step_paired
INFO [step step_paired] start
INFO [job step_paired] /private/tmp/docker_tmpbu8okwlo$ mv \
    /private/var/folders/ps/xwjn82ms7qz1s2p6qbywmlt4v7qk6n/T/tmp_jaz7j5w/stg4b01e05b-343a-4661-ae88-48f7069dd36a/test.file \
    filename_paired
INFO [job step_paired] completed success
INFO [step step_paired] start
INFO [job step_paired_2] /private/tmp/docker_tmpcd2e7p0m$ mv \
    /private/var/folders/ps/xwjn82ms7qz1s2p6qbywmlt4v7qk6n/T/tmpq0h7oov4/stg04845da8-4f0c-4f28-ae8b-e25d7d5428f1/test.file \
    filename_paired
INFO [job step_paired_2] completed success
INFO [step step_paired] completed success
INFO [workflow ] completed success
{}
INFO Final process status is success

Unfortunately, the same wf doesn’t work with toil (full log attached): toil-cwl-runner --preserve-entire-environment --enable-dev --disableChaining --logFile log --jobStore job-store --outdir output --no-container --batchSystem LSF --disableCaching --defaultMemory 5G --defaultCores 8 --retryCount 3 wf.cwl wf.yml Toil version 4.2.0a1-3f4e790c1fa32a2c2257bd5cd203a3b51d5d5661

Log from job kind-file_hps_nobackup2_production_metagenomics_pipeline_testing_kate_pipeline-v5_workflows_test_mv.cwl/instance-l3053lvf follows:
=========>
        INFO:toil.worker:---TOIL WORKER OUTPUT LOG---
        INFO:toil:Running Toil version 4.2.0a1-3f4e790c1fa32a2c2257bd5cd203a3b51d5d5661.
        Traceback (most recent call last):
          File "/hps/nobackup/production/metagenomics/software/toil-v4/v3nv/lib/python3.6/site-packages/cwltool/process.py", line 455, in fill_in_defaults
            "Missing required input parameter '%s'" % shortname(inp["id"])
        cwltool.errors.WorkflowException: Missing required input parameter 'initial_file'
        
        The above exception was the direct cause of the following exception:
        
        Traceback (most recent call last):
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/worker.py", line 368, in workerScript
            job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/job.py", line 1392, in _runner
            returnValues = self._run(jobGraph, fileStore)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/job.py", line 1329, in _run
            return self.run(fileStore)
          File "/hps/nobackup/production/metagenomics/software/toil-v4/src/toil/cwl/cwltoil.py", line 938, in run
            self.runtime_context.make_fs_access(""))
          File "/hps/nobackup/production/metagenomics/software/toil-v4/v3nv/lib/python3.6/site-packages/cwltool/process.py", line 455, in fill_in_defaults
            "Missing required input parameter '%s'" % shortname(inp["id"])

test.zip

┆Issue is synchronized with this Jira Task ┆Issue Number: TOIL-575

[mr-c]

I think I’ve implemented @tetron 's suggestion correctly at https://github.com/DataBiosphere/toil/pull/3117 but I need to develop and test a minimal example before knowing for sure. Just wanted to get this out there for @KeteSakharova to use

[mr-c]

@KeteSakharova Nevermind, I think I figured it out -> https://github.com/DataBiosphere/toil/pull/3117#issuecomment-660902347