Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
cwltoil - does not read valueFrom correctly when scattering steps
See original GitHub issueI’m attempting to run a cwl workflow with cwltoil that uses ScatterFeatureRequirement and StepInputExpressionRequirement. For a few of the step inputs (not scattered), the value is wired to a valueFrom: constant. This causes the job to fail under cwltoil. The valueFrom expression does not make it into the job dictionary, and fillIInDefaults() raises an exception when the input is missing.
The job runs successfully under cwltool (1.0.20160425140546)
I have a simple test case in https://github.com/dleehr/broken-valueFrom and can confirm it only fails when scattering:
Working: https://github.com/dleehr/broken-valueFrom/tree/1c3ff9be2a353ea8ac5cc0d95e9a61ebe89514f3
$ cwltoil --logDebug echo-workflow.cwl echo.json
Homer15.local: 2016-05-26 11:41:27,454 INFO: toil.jobStores.fileJobStore: Jobstore directory is: /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpu0PvFJ
Homer15.local: 2016-05-26 11:41:27,454 INFO: toil.jobStores.abstractJobStore: The workflow ID is: 'a6bb977d-dbcd-4c72-9d24-6b8952090dc8'
Homer15.local: 2016-05-26 11:41:27,461 INFO: toil.common: Using the single machine batch system
Homer15.local: 2016-05-26 11:41:27,461 WARNING: toil.batchSystems.singleMachine: Limiting maxCores to CPU count of system (8).
Homer15.local: 2016-05-26 11:41:27,461 WARNING: toil.batchSystems.singleMachine: Limiting maxMemory to physically available memory (17179869184).
Homer15.local: 2016-05-26 11:41:27,462 INFO: toil.batchSystems.singleMachine: Setting up the thread pool with 80 workers, given a minimum CPU fraction of 0.100000 and a maximum CPU value of 8.
Homer15.local: 2016-05-26 11:41:27,470 INFO: toil.common: Written the environment for the jobs to the environment file
Homer15.local: 2016-05-26 11:41:27,470 INFO: toil.common: Caching all jobs in job store
Homer15.local: 2016-05-26 11:41:27,470 INFO: toil.common: 0 jobs downloaded.
Homer15.local: 2016-05-26 11:41:27,514 INFO: toil.realtimeLogger: Real-time logging disabled
Homer15.local: 2016-05-26 11:41:27,516 INFO: toil.leader: (Re)building internal scheduler state
Homer15.local: 2016-05-26 11:41:27,516 INFO: toil.leader: Checked batch system has no running jobs and no updated jobs
Homer15.local: 2016-05-26 11:41:27,516 INFO: toil.leader: Found 1 jobs to start and 0 jobs with successors to run
Homer15.local: 2016-05-26 11:41:27,519 INFO: toil.leader: Starting the main loop
Homer15.local: 2016-05-26 11:41:27,521 INFO: toil.batchSystems.singleMachine: Executing command: '/Users/dcl9/Code/python/toil/env/bin/_toil_worker /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpu0PvFJ f/h/job1ELvHk'.
INFO:toil.common:Created the workflow directory at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-a6bb977d-dbcd-4c72-9d24-6b8952090dc8
Homer15.local: 2016-05-26 11:41:27,872 INFO: toil.leader: No jobs left to run so exiting.
Homer15.local: 2016-05-26 11:41:27,872 INFO: toil.leader: Finished the main loop
Homer15.local: 2016-05-26 11:41:27,872 INFO: toil.leader: Waiting for stats and logging collator process to finish ...
Homer15.local: 2016-05-26 11:41:28,027 INFO: toil.leader: f/h/job1ELvHk: ---TOIL WORKER OUTPUT LOG---
Homer15.local: 2016-05-26 11:41:28,028 INFO: toil.leader: f/h/job1ELvHk: Next available file descriptor: 10
Homer15.local: 2016-05-26 11:41:28,028 INFO: toil.leader: f/h/job1ELvHk: Parsed jobWrapper
Homer15.local: 2016-05-26 11:41:28,029 INFO: toil.leader: f/h/job1ELvHk: Got a command to run: _toil f/h/job1ELvHk/g/tmpMMNqCx.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:41:28,029 INFO: toil.leader: f/h/job1ELvHk: INFO:rdflib:RDFLib Version: 4.2.1
Homer15.local: 2016-05-26 11:41:28,029 INFO: toil.leader: f/h/job1ELvHk: Starting the next jobWrapper
Homer15.local: 2016-05-26 11:41:28,029 INFO: toil.leader: f/h/job1ELvHk: Got a command to run: _toil f/h/job1ELvHk/g/tmpOGqAhj.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:41:28,030 INFO: toil.leader: f/h/job1ELvHk: [job 4516264720] /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-a6bb977d-dbcd-4c72-9d24-6b8952090dc8/tmpuoyH4H/localTempDir/tlziRI1/out$ echo 'hello from the workflow'
Homer15.local: 2016-05-26 11:41:28,030 INFO: toil.leader: f/h/job1ELvHk: INFO:cwltool:[job 4516264720] /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-a6bb977d-dbcd-4c72-9d24-6b8952090dc8/tmpuoyH4H/localTempDir/tlziRI1/out$ echo 'hello from the workflow'
Homer15.local: 2016-05-26 11:41:28,030 INFO: toil.leader: f/h/job1ELvHk: hello from the workflow
Homer15.local: 2016-05-26 11:41:28,030 INFO: toil.leader: f/h/job1ELvHk: Final process status is success
Homer15.local: 2016-05-26 11:41:28,030 INFO: toil.leader: f/h/job1ELvHk: INFO:cwltool:Final process status is success
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: INFO:toil.jobStores.fileJobStore:Jobstore directory is: /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpu0PvFJ
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: Starting the next jobWrapper
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: Got a command to run: _toil f/h/job1ELvHk/g/tmpoXUfR6.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: Stopping running chain of jobs: length of stack: 0, services: 0, checkpoint: False
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: Worker log can be found at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-a6bb977d-dbcd-4c72-9d24-6b8952090dc8/tmpuoyH4H/localTempDir. Set --cleanWorkDir to retain this log
Homer15.local: 2016-05-26 11:41:28,031 INFO: toil.leader: f/h/job1ELvHk: INFO:toil.worker:Worker log can be found at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-a6bb977d-dbcd-4c72-9d24-6b8952090dc8/tmpuoyH4H/localTempDir. Set --cleanWorkDir to retain this log
Homer15.local: 2016-05-26 11:41:28,032 INFO: toil.leader: f/h/job1ELvHk: Finished running the chain of jobs on this node, we ran for a total of 0.149798 seconds
Homer15.local: 2016-05-26 11:41:28,032 INFO: toil.leader: f/h/job1ELvHk: INFO:toil.worker:Finished running the chain of jobs on this node, we ran for a total of 0.149798 seconds
Homer15.local: 2016-05-26 11:41:28,035 INFO: toil.leader: ... finished collating stats and logs. Took 0.162292957306 seconds
Homer15.local: 2016-05-26 11:41:28,035 INFO: toil.leader: Waiting for service manager thread to finish ...
Homer15.local: 2016-05-26 11:41:28,518 INFO: toil.leader: ... finished shutting down the service manager. Took 0.482532024384 seconds
Homer15.local: 2016-05-26 11:41:28,518 INFO: toil.leader: Finished toil run successfully
Homer15.local: 2016-05-26 11:41:28,539 INFO: toil.common: Attempting to delete the job store
Homer15.local: 2016-05-26 11:41:28,540 INFO: toil.common: Successfully deleted the job store
Broken: https://github.com/dleehr/broken-valueFrom/tree/de36eb884fef12b37fb78dac7c0c5f034f2ad34a
$ cwltoil --logDebug echo-workflow.cwl echo.json
Homer15.local: 2016-05-26 11:46:00,433 INFO: toil.jobStores.fileJobStore: Jobstore directory is: /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpr5YgBI
Homer15.local: 2016-05-26 11:46:00,434 INFO: toil.jobStores.abstractJobStore: The workflow ID is: '8658e2b4-fe86-4a94-b472-60aa5fb8eb05'
Homer15.local: 2016-05-26 11:46:00,441 INFO: toil.common: Using the single machine batch system
Homer15.local: 2016-05-26 11:46:00,441 WARNING: toil.batchSystems.singleMachine: Limiting maxCores to CPU count of system (8).
Homer15.local: 2016-05-26 11:46:00,441 WARNING: toil.batchSystems.singleMachine: Limiting maxMemory to physically available memory (17179869184).
Homer15.local: 2016-05-26 11:46:00,441 INFO: toil.batchSystems.singleMachine: Setting up the thread pool with 80 workers, given a minimum CPU fraction of 0.100000 and a maximum CPU value of 8.
Homer15.local: 2016-05-26 11:46:00,449 INFO: toil.common: Written the environment for the jobs to the environment file
Homer15.local: 2016-05-26 11:46:00,449 INFO: toil.common: Caching all jobs in job store
Homer15.local: 2016-05-26 11:46:00,449 INFO: toil.common: 0 jobs downloaded.
Homer15.local: 2016-05-26 11:46:00,488 INFO: toil.realtimeLogger: Real-time logging disabled
Homer15.local: 2016-05-26 11:46:00,489 INFO: toil.leader: (Re)building internal scheduler state
Homer15.local: 2016-05-26 11:46:00,490 INFO: toil.leader: Checked batch system has no running jobs and no updated jobs
Homer15.local: 2016-05-26 11:46:00,490 INFO: toil.leader: Found 1 jobs to start and 0 jobs with successors to run
Homer15.local: 2016-05-26 11:46:00,492 INFO: toil.leader: Starting the main loop
Homer15.local: 2016-05-26 11:46:00,494 INFO: toil.batchSystems.singleMachine: Executing command: '/Users/dcl9/Code/python/toil/env/bin/_toil_worker /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpr5YgBI 1/A/job9KPMku'.
INFO:toil.common:Created the workflow directory at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-8658e2b4-fe86-4a94-b472-60aa5fb8eb05
Homer15.local: 2016-05-26 11:46:00,930 INFO: toil.batchSystems.singleMachine: Executing command: '/Users/dcl9/Code/python/toil/env/bin/_toil_worker /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpr5YgBI P/I/job4QGgEc'.
Homer15.local: 2016-05-26 11:46:00,933 INFO: toil.batchSystems.singleMachine: Executing command: '/Users/dcl9/Code/python/toil/env/bin/_toil_worker /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpr5YgBI S/8/jobEIUZqs'.
Homer15.local: 2016-05-26 11:46:00,999 INFO: toil.leader: 1/A/job9KPMku: ---TOIL WORKER OUTPUT LOG---
Homer15.local: 2016-05-26 11:46:00,999 INFO: toil.leader: 1/A/job9KPMku: Next available file descriptor: 10
Homer15.local: 2016-05-26 11:46:01,000 INFO: toil.leader: 1/A/job9KPMku: Parsed jobWrapper
Homer15.local: 2016-05-26 11:46:01,000 INFO: toil.leader: 1/A/job9KPMku: Got a command to run: _toil 1/A/job9KPMku/g/tmphxCjaN.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:46:01,000 INFO: toil.leader: 1/A/job9KPMku: INFO:rdflib:RDFLib Version: 4.2.1
Homer15.local: 2016-05-26 11:46:01,000 INFO: toil.leader: 1/A/job9KPMku: Starting the next jobWrapper
Homer15.local: 2016-05-26 11:46:01,001 INFO: toil.leader: 1/A/job9KPMku: Got a command to run: _toil 1/A/job9KPMku/g/tmp2mY5of.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:46:01,001 INFO: toil.leader: 1/A/job9KPMku: No more jobs can run in series by this worker, it's got 1 children
Homer15.local: 2016-05-26 11:46:01,002 INFO: toil.leader: 1/A/job9KPMku: Worker log can be found at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-8658e2b4-fe86-4a94-b472-60aa5fb8eb05/tmpeY5Xir/localTempDir. Set --cleanWorkDir to retain this log
Homer15.local: 2016-05-26 11:46:01,002 INFO: toil.leader: 1/A/job9KPMku: INFO:toil.worker:Worker log can be found at /var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/toil-8658e2b4-fe86-4a94-b472-60aa5fb8eb05/tmpeY5Xir/localTempDir. Set --cleanWorkDir to retain this log
Homer15.local: 2016-05-26 11:46:01,002 INFO: toil.leader: 1/A/job9KPMku: Finished running the chain of jobs on this node, we ran for a total of 0.226353 seconds
Homer15.local: 2016-05-26 11:46:01,002 INFO: toil.leader: 1/A/job9KPMku: INFO:toil.worker:Finished running the chain of jobs on this node, we ran for a total of 0.226353 seconds
Exception in thread Thread-2:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 528, in asyncWrite
raise RuntimeError("The termination flag is set, exiting")
RuntimeError: The termination flag is set, exiting
Exception in thread Thread-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 528, in asyncWrite
raise RuntimeError("The termination flag is set, exiting")
RuntimeError: The termination flag is set, exiting
Exception RuntimeError: RuntimeError('cannot join current thread',) in <bound method FileStore.__del__ of <toil.job.FileStore object at 0x10530e410>> ignored
Exception in thread Thread-2:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 528, in asyncWrite
raise RuntimeError("The termination flag is set, exiting")
RuntimeError: The termination flag is set, exiting
Exception in thread Thread-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 528, in asyncWrite
raise RuntimeError("The termination flag is set, exiting")
RuntimeError: The termination flag is set, exiting
Exception RuntimeError: RuntimeError('cannot join current thread',) in <bound method FileStore.__del__ of <toil.job.FileStore object at 0x1015cc410>> ignored
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: The jobWrapper seems to have left a log file, indicating failure: P/I/job4QGgEc
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: Reporting file: P/I/job4QGgEc
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: ---TOIL WORKER OUTPUT LOG---
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: Next available file descriptor: 10
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: Parsed jobWrapper
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: Got a command to run: _toil 1/A/job9KPMku/g/tmpdVH1yz.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: INFO:rdflib:RDFLib Version: 4.2.1
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: Traceback (most recent call last):
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: File "/Users/dcl9/Code/python/toil/src/toil/worker.py", line 327, in main
Homer15.local: 2016-05-26 11:46:03,224 WARNING: toil.leader: P/I/job4QGgEc: fileStore=fileStore)
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 1384, in _execute
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: returnValues = self._run(jobWrapper, fileStore)
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 1373, in _run
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: return self.run(fileStore)
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: File "/Users/dcl9/Code/python/toil/src/toil/cwl/cwltoil.py", line 199, in run
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: fillInDefaults(self.cwltool.tool["inputs"], cwljob)
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: File "/Users/dcl9/Code/python/toil/env/lib/python2.7/site-packages/cwltool/process.py", line 205, in fillInDefaults
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: raise validate.ValidationException("Missing input parameter `%s`" % shortname(inp["id"]))
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: ValidationException: Missing input parameter `message`
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: Exiting the worker because of a failed jobWrapper on host Homer15.local
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: ERROR:toil.worker:Exiting the worker because of a failed jobWrapper on host Homer15.local
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: WARNING:toil.jobWrapper:Due to failure we are reducing the remaining retry count of job P/I/job4QGgEc to 0
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: P/I/job4QGgEc: WARNING:toil.jobWrapper:We have increased the default memory of the failed job to 2147483648 bytes
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: Job: P/I/job4QGgEc is completely failed
Homer15.local: 2016-05-26 11:46:03,225 WARNING: toil.leader: The jobWrapper seems to have left a log file, indicating failure: S/8/jobEIUZqs
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: Reporting file: S/8/jobEIUZqs
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: ---TOIL WORKER OUTPUT LOG---
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: Next available file descriptor: 10
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: Parsed jobWrapper
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: Got a command to run: _toil 1/A/job9KPMku/g/tmpurGddE.tmp /Users/dcl9/Code/python/toil/src toil.cwl.cwltoil
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: INFO:rdflib:RDFLib Version: 4.2.1
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: Traceback (most recent call last):
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: File "/Users/dcl9/Code/python/toil/src/toil/worker.py", line 327, in main
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: fileStore=fileStore)
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 1384, in _execute
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: returnValues = self._run(jobWrapper, fileStore)
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: File "/Users/dcl9/Code/python/toil/src/toil/job.py", line 1373, in _run
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: return self.run(fileStore)
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: File "/Users/dcl9/Code/python/toil/src/toil/cwl/cwltoil.py", line 199, in run
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: fillInDefaults(self.cwltool.tool["inputs"], cwljob)
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: File "/Users/dcl9/Code/python/toil/env/lib/python2.7/site-packages/cwltool/process.py", line 205, in fillInDefaults
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: raise validate.ValidationException("Missing input parameter `%s`" % shortname(inp["id"]))
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: ValidationException: Missing input parameter `message`
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: Exiting the worker because of a failed jobWrapper on host Homer15.local
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: ERROR:toil.worker:Exiting the worker because of a failed jobWrapper on host Homer15.local
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: WARNING:toil.jobWrapper:Due to failure we are reducing the remaining retry count of job S/8/jobEIUZqs to 0
Homer15.local: 2016-05-26 11:46:03,226 WARNING: toil.leader: S/8/jobEIUZqs: WARNING:toil.jobWrapper:We have increased the default memory of the failed job to 2147483648 bytes
Homer15.local: 2016-05-26 11:46:03,227 WARNING: toil.leader: Job: S/8/jobEIUZqs is completely failed
Homer15.local: 2016-05-26 11:46:05,230 INFO: toil.leader: No jobs left to run so exiting.
Homer15.local: 2016-05-26 11:46:05,230 INFO: toil.leader: Finished the main loop
Homer15.local: 2016-05-26 11:46:05,230 INFO: toil.leader: Waiting for stats and logging collator process to finish ...
Homer15.local: 2016-05-26 11:46:05,521 INFO: toil.leader: ... finished collating stats and logs. Took 0.291285991669 seconds
Homer15.local: 2016-05-26 11:46:05,522 INFO: toil.leader: Waiting for service manager thread to finish ...
Homer15.local: 2016-05-26 11:46:06,503 INFO: toil.leader: ... finished shutting down the service manager. Took 0.981132030487 seconds
Homer15.local: 2016-05-26 11:46:06,503 INFO: toil.leader: Finished toil run with 3 failed jobs
Homer15.local: 2016-05-26 11:46:06,503 INFO: toil.leader: Failed jobs at end of the run: set(['S/8/jobEIUZqs', '1/A/job9KPMku', 'P/I/job4QGgEc'])
Traceback (most recent call last):
File "/Users/dcl9/Code/python/toil/env/bin/cwltoil", line 9, in <module>
load_entry_point('toil', 'console_scripts', 'cwltoil')()
File "/Users/dcl9/Code/python/toil/src/toil/cwl/cwltoil.py", line 595, in main
outobj = toil.start(wf1)
File "/Users/dcl9/Code/python/toil/src/toil/common.py", line 529, in start
return self._runMainLoop(job)
File "/Users/dcl9/Code/python/toil/src/toil/common.py", line 740, in _runMainLoop
jobCache=self._jobCache)
File "/Users/dcl9/Code/python/toil/src/toil/leader.py", line 689, in mainLoop
raise FailedJobsException( config.jobStore, len(toilState.totalFailedJobs) )
toil.leader.FailedJobsException: The job store '/var/folders/7t/q8bb2np92p5b26pk_h5lr0lh0000gn/T/tmpr5YgBI' contains 3 failed jobs
Issue Analytics
- State:
- Created 7 years ago
- Comments:16 (16 by maintainers)
Top Results From Across the Web
toil/cwltoil.py at master · DataBiosphere/toil - GitHub
A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python. - toil/cwltoil.py at master · DataBiosphere/toil.
Read more >Toil Documentation
Toil is an open-source pure-Python workflow engine that lets people write better pipelines. Check out our website for a comprehensive list ...
Read more >Ubuntu Manpage: toil - Toil Documentation
Running a basic workflow A Toil workflow can be run with just two steps: 1. ... The job doesn't actually get run until...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hannes – thanks so much for merging the restart PR and managing everything.
I pushed an in-progress PR (#1340) which has my current attempt to synchronize with the latest cwltool. Things mostly work but I’m failing 5 conformance tests and added notes about where I’m stuck with those. If Peter or Michael have any ideas or pointers that would be so helpful to keep moving. Thanks much.
I just merged @chapmanb’s #1107 about
--restartwith cwltoil. The only other CWL-related tickets I am aware of areTL;DR: There is no branch to continue work on. Master is more or less up-to-date.