cwltool pull of node:slim violates job timeout

cwltool version 1.0.20170413194156

The symptom I found was this output:

Traceback (most recent call last):
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/workflow.py", line 391, in job
    for newjob in step.iterable:
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/workflow.py", line 338, in try_make_job
    for j in jobs:
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/workflow.py", line 204, in job
    for j in self.step.job(joborder, output_callback, **kwargs):
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/workflow.py", line 587, in job
    **kwargs):
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/draft2tool.py", line 384, in job
    j.command_line = flatten(map(builder.generate_arg, builder.bindings))
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/cwltool/builder.py", line 175, in generate_arg
    value = self.do_eval(binding["valueFrom"], context=value)
  File "/cip0/software/x86_64/miniconda3-4.3.11/envs/cwltool/lib/python2.7/site-packages/schema_salad/sourceline.py", line 150, in __exit__
    raise self.makeError(unicode(exc_value))
WorkflowException: :1:1: Expression evaluation error:
:1:1: Long-running script killed after 20 seconds: returncode was: -9
:1:1: script was:
:1:1: 01 "use strict";
:1:1: 02 var new_ext = function() { var ext=inputs.bai?'.bai':inputs.csi?'.csi':'.bai'; return inputs.input.path.split('/').slice(-1)[0].replace(/.bam$/, '')+ext; };
:1:1: 03 var inputs = {
:1:1: 04     "input": {
:1:1: 05         "format": "http://edamontology.org/format_2572", 
:1:1: 06         "checksum": "sha1$4b1115981b3127f44204a2bb1252d6c9e06a15a2", 
:1:1: 07         "basename": "samtools-sort-2017-04-23.bam", 
:1:1: 08         "nameroot": "samtools-sort-2017-04-23", 
:1:1: 09         "nameext": ".bam", 
:1:1: 10         "location": "file:///cip0/research/scratch/pvh/h3agatk/tmp/workingI32Emb/samtools-sort-2017-04-23.bam", 
:1:1: 11         "path": "/var/lib/cwl/stgd37e2525-45bf-4b3c-8c5f-83d6393090f7/samtools-sort-2017-04-23.bam", 
:1:1: 12         "dirname": "/var/lib/cwl/stgd37e2525-45bf-4b3c-8c5f-83d6393090f7", 
:1:1: 13         "class": "File", 
:1:1: 14         "size": 11741446772
:1:1: 15     }, 
:1:1: 16     "interval": null, 
:1:1: 17     "csi": false, 
:1:1: 18     "bai": true
:1:1: 19 };
:1:1: 20 var self = null;
:1:1: 21 var runtime = {
:1:1: 22     "outdirSize": 1024, 
:1:1: 23     "ram": 1024, 
:1:1: 24     "tmpdirSize": 1024, 
:1:1: 25     "cores": 1, 
:1:1: 26     "tmpdir": "/tmp", 
:1:1: 27     "outdir": "/home/spool/cwl"
:1:1: 28 };
:1:1: 29 (function(){return ((inputs.bai?'-b':inputs.csi?'-c':[]));})()
:1:1: stdout was: 
:1:1: stderr was: 
:1:1: Unable to find image 'node:slim' locally
:1:1: slim: Pulling from library/node
:1:1: 10a267c67f42: Pulling fs layer
:1:1: fb5937da9414: Pulling fs layer
:1:1: 7c4b8c9fa331: Pulling fs layer
:1:1: d31a861b11c6: Pulling fs layer
:1:1: 016d32b8ca5f: Pulling fs layer
:1:1: a0d498472cf0: Pulling fs layer
:1:1: d31a861b11c6: Waiting
:1:1: 016d32b8ca5f: Waiting
:1:1: a0d498472cf0: Waiting
:1:1: 7c4b8c9fa331: Download complete
:1:1: d31a861b11c6: Download complete
:1:1: fb5937da9414: Verifying Checksum
:1:1: fb5937da9414: Download complete
:1:1: a0d498472cf0: Verifying Checksum
:1:1: a0d498472cf0: Download complete

What seems to have happened is that the docker pull of node:slim led to the timeout (20 seconds) for Javascript procedure being violated. A possible solution might be to implement the check for node and docker pull logic outside new_js_proc() with a timeout independent of the Javascript procedure timeout.

[pvanheus]

@kapilkd13 docker images -q node:slim gives output (the image ID) if the image exists, nothing if it doesn’t exist. It is thus a better test to use than docker images.

The purpose of new_js_proc() is to return a Popen object that either refers to a locally executed nodejs or to nodejs running in a Docker container. So it should not be blocking.

[manu-chroma]

sure, I will make a PR for this one 👍 Thanks!