cwltoil does not pass setEnv variables to cwl job

When running cwltoil with the --no-container option, jobs fail if the specified CWL baseCommand is not found in the runtime PATH. This is expected, but it seems there’s no way to provide that path to cwltoil. There is a --setEnv argument that appears to be for this purpose (setting environment variables on jobs), but cwltoil does not honor it.

Running cwltool, the environment (including PATH) is inherited from the parent process/shell, so binaries specified in baseCommand are found if they’re in the PATH.

cwltool command:

PATH=$PATH:/home/dcl9/bin cwltool --no-container --outdir /home/dcl9/ggr-output/ /home/dcl9/GGR-cwl/qc/fastqc.cwl fastqc_job.json

results: cwltool-succeed.txt

cwltoil command:

cwltoil --jobStore /data/tmp/toil-jobstore/ --no-container --setEnv PATH=/home/dcl9/bin --outdir /home/dcl9/ggr-output/ /home/dcl9/GGR-cwl/qc/fastqc.cwl fastqc_job.json

results: cwltoil-fail.txt

2016-05-02 10:11:58,668 WARNING:toil.leader: P/p/jobGD9RLU: [job 28436624] /tmp/toil-24191727-ee15-4522-93b3-6be6f484b9a6/tmpIsrHjR/localTempDir/tyO2hO6/out$ fastqc -o /var/spool/cwl --noextract --format fastq /var/lib/cwl/job378892700_tmpSLsWRh/SRC1_ab2859_GR220236_6_160309_S18.tX_repX.fastq --threads 1 --dir /tmp
2016-05-02 10:11:58,668 WARNING:toil.leader: P/p/jobGD9RLU: INFO:cwltool:[job 28436624] /tmp/toil-24191727-ee15-4522-93b3-6be6f484b9a6/tmpIsrHjR/localTempDir/tyO2hO6/out$ fastqc -o /var/spool/cwl --noextract --format fastq /var/lib/cwl/job378892700_tmpSLsWRh/SRC1_ab2859_GR220236_6_160309_S18.tX_repX.fastq --threads 1 --dir /tmp
2016-05-02 10:11:58,668 WARNING:toil.leader: P/p/jobGD9RLU: 'fastqc' not found
2016-05-02 10:11:58,669 WARNING:toil.leader: P/p/jobGD9RLU: ERROR:cwltool:'fastqc' not found
2016-05-02 10:11:58,669 WARNING:toil.leader: P/p/jobGD9RLU: [job 28436624] completed permanentFail

fastqc_job.json:

{"input_fastq_file":{ "class": "File", "path": "/data/projects/GGR/data/chip_seq/processed_raw_reads/iter-1/SRC1_ab2859_GR220236_6_160309_S18.tX_repX.fastq" }}

fastqc.cwl: https://github.com/Duke-GCB/GGR-cwl/blob/master/qc/fastqc.cwl

[dleehr]

I’m tinkering with this in a branch and made some progress today.

It doesn’t seem to be a problem with --setEnv actually, we just need some CLI arguments propagated to CWLJob.run().

CWLJob.run() calls cwltool.main.single_job_executor(), which eventually calls down to cwltool.job.run() with some kwargs (outdir, tmpdir, use_container).

When I include preserve-environment and set use_container to False, my above fastqc workflow runs.

I’ll take a crack at passing preserve-environment along (defaulting to ["PATH"] as cwltool does), as well as honoring no-container.

[dleehr]

@guillermo-carrasco That looks like exactly what we need, great!