Error "RuntimeError: close method on generator classes unimplemented" is reported when "--parallel" is used

workflow:

class: Workflow
cwlVersion: v1.0
id: test
label: test
$namespaces:
  sbg: 'https://www.sevenbridges.com/'
inputs:
  - id: sample
    type: 'string[]?'
    default:
      - sample1
    'sbg:x': 0
    'sbg:y': 532.4375
  - id: fastq1
    type: 'File[]'
    default:
      - class: File
        path: ./sample1.raw_1.fq.gz
    'sbg:x': 0
    'sbg:y': 853.015625
  - id: fastq2
    type: 'File[]?'
    default:
      - class: File
        path: ./sample1.raw_2.fq.gz
    'sbg:x': 0
    'sbg:y': 746.15625
outputs: []
steps:
  - id: cutadaptparallel_v2
    in:
      - id: fastq1
        source:
          - fastq1
      - id: fastq2
        source:
          - fastq2
      - id: sample
        source:
          - sample
    out:
      - id: outfastq1
      - id: outfastq2
    run:
      class: Workflow
      cwlVersion: v1.0
      id: cutadaptparallel_v2
      label: cutadaptparallel_v2
      $namespaces:
        sbg: 'https://www.sevenbridges.com/'
      inputs:
        - id: fastq1
          type: 'File[]?'
          'sbg:x': -323.37908935546875
          'sbg:y': -315.423583984375
        - id: fastq2
          type: 'File[]?'
          'sbg:x': -323.4378662109375
          'sbg:y': -445.9402160644531
        - id: sample
          type: 'string[]?'
          'sbg:x': -324.03839111328125
          'sbg:y': -556.1343383789062
      outputs:
        - id: outfastq1
          outputSource:
            - cutadapt_v2/outfastq1
          type: 'File[]?'
          'sbg:x': 90.7265625
          'sbg:y': -422
        - id: outfastq2
          outputSource:
            - cutadapt_v2/outfastq2
          type: 'File[]?'
          'sbg:x': 36
          'sbg:y': -564
      steps:
        - id: cutadapt_v2
          in:
            - id: fastq1
              source: fastq1
            - id: fastq2
              source: fastq2
            - id: sample
              source: sample
          out:
            - id: outfastq1
            - id: outfastq2
          run:
            class: CommandLineTool
            cwlVersion: v1.0
            $namespaces:
              sbg: 'https://www.sevenbridges.com/'
            id: cutadapt_v2
            baseCommand:
              - cutadapt
            inputs:
              - id: fastq1
                type: File
                inputBinding:
                  position: 4
              - default: AGATCGGAAGAGC
                id: adapter
                type: string?
                inputBinding:
                  position: 1
                  prefix: '-b'
              - default: $(inputs.sample).cutadapt.fq1.gz
                id: output
                type: string?
                inputBinding:
                  position: 2
                  prefix: '-o'
                  valueFrom: $(inputs.sample).cutadapt.fq1.gz
              - default: $(inputs.sample).cutadapt.fq2.gz
                id: paired_output
                type: string?
                inputBinding:
                  position: 3
                  prefix: '-p'
                  valueFrom: $(inputs.sample).cutadapt.fq2.gz
              - id: fastq2
                type: File?
                inputBinding:
                  position: 5
              - id: sample
                type: string?
            outputs:
              - id: outfastq1
                type: File?
                outputBinding:
                  glob: |
                    $(inputs.sample).cutadapt.fq1.gz
              - id: outfastq2
                type: File?
                outputBinding:
                  glob: |
                    $(inputs.sample).cutadapt.fq2.gz
            doc: >-
              cutadapt  -b AGATCGGAAGAGC  -o 00.rawdata/sample1.raw_1.fq -p
              00.rawdata/sample1.raw_2.fq 
              /gluster/home/micro/Pipeline/16S_pipeline_V3.0/example/00.data/sample1.raw_1.fq.gz 
              /gluster/home/micro/Pipeline/16S_pipeline_V3.0/example/00.data/sample1.raw_2.fq.gz
              >00.rawdata/cutadapt.log
            label: cutadapt-v2
            hints:
              - class: DockerRequirement
                dockerPull: 'cutadapt:1.16'
            requirements:
              - class: InlineJavascriptRequirement
          label: cutadapt-v2
          scatter:
            - fastq1
            - fastq2
            - sample
          scatterMethod: dotproduct
          'sbg:x': -88.69266510009766
          'sbg:y': -470.6273193359375
      requirements:
        - class: ScatterFeatureRequirement
    label: cutadaptparallel_v2
    'sbg:x': 540.9274291992188
    'sbg:y': 677.3812255859375
requirements:
  - class: SubworkflowFeatureRequirement

If the image is created using “cwltool.Dockerfile”, the following error message is displayed:

INFO [job cutadapt_v2] Max memory used: 6MiB
INFO [job cutadapt_v2] completed success
INFO [step cutadapt_v2] completed success
INFO [workflow cutadaptparallel_v2] completed success
INFO [step cutadaptparallel_v2] completed success
ERROR Unhandled exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/cwltool/workflow_job.py", line 777, in try_make_job
    yield from jobs
  File "/usr/local/lib/python3.10/site-packages/cwltool/workflow_job.py", line 76, in job
    yield from self.step.job(joborder, output_callback, runtimeContext)
RuntimeError: close method on generator classes unimplemented

However, if use cwLTool installed directly by Conda, it will work without error

The file used for the test: test.zip

[mr-c]

I was able to reproduce this problem using mypy 0.942 and with mypy 0.950 it is fixed; I’ll make a new cwltool release so that others aren’t affected. Thanks!

[kongxiangya]

I have tested version 3.1.20220502060230 and SET CWLTOOL_USE_MYPYC=1 without any error. Thanks.