Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Toil throws KeyError after workflow execution ends

See original GitHub issue

I tried running toil on an RNA-Seq workflow, accessible at https://github.com/FarahZKhan/cwl_workflows/blob/master/topmed-workflows/TOPMed_RNAseq_pipeline/rnaseq_pipeline_fastq.cwl

The workflow runs smoothly and produces all relevant files(I cross checked by running same pipeline with cwltool). But it crashes as soon as workflow execution ends.

Here’s the log

  File "/home/prakash/.conda/envs/toil/bin/toil-cwl-runner", line 8, in <module>
    sys.exit(main())
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 1702, in main
    destBucket=options.destBucket)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 738, in toilStageFiles
    visit_class(cwljob, ("File", "Directory"), _check_adjust)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/cwltool/utils.py", line 231, in visit_class
    visit_class(rec[d], cls, op)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/cwltool/utils.py", line 231, in visit_class
    visit_class(rec[d], cls, op)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/cwltool/utils.py", line 234, in visit_class
    visit_class(d, cls, op)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/cwltool/utils.py", line 229, in visit_class
    op(rec)
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 733, in _check_adjust
    pm.mapper(f["location"])[1])
  File "/home/prakash/.conda/envs/toil/lib/python3.6/site-packages/cwltool/pathmapper.py", line 385, in mapper
    return self._pathmap[src]
KeyError: 'toilfs:89664:files/for-job/kind-file_mnt_disk8tb_prakash_cwl_workflows_topmed-workflows_TOPMed_RNAseq_pipeline_markduplicates.cwl_run-MarkDuplicates/instance-p59fm9hc/file-21j7zgac/LC_C13_cRNA.Aligned.sortedByCoord.out.md.bam.bai'

Any clue on what could’ve gone wrong?

┆Issue is synchronized with this Jira Task ┆Issue Number: TOIL-585