CWLGather - still RuntimeError: This job was passed a promise that wasn't yet resolved when it ran.

Despite #2221, I am still getting the errors (below). My workflow contains ExpressionTool upstream of that gather. Are ExpressionTools supported by Toil?

8/V/jobLgOhDM    WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'ResolveIndirect' 8/V/jobLgOhDM with ID 8/V/jobLgOhDM to 5
The job seems to have left a log file, indicating failure: 'ResolveIndirect' 8/V/jobLgOhDM
8/V/jobLgOhDM    INFO:toil.worker:---TOIL WORKER OUTPUT LOG---
8/V/jobLgOhDM    INFO:toil:Running Toil version 3.17.0a1-7eff049a77c125e29de58b70a1cb388c7f1cbc5b.
8/V/jobLgOhDM    Traceback (most recent call last):
8/V/jobLgOhDM      File ".../toil/src/toil/worker.py", line 298, in workerScript
8/V/jobLgOhDM        job = Job._loadJob(jobGraph.command, jobStore)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 917, in _loadJob
8/V/jobLgOhDM        return cls._unpickle(userModule, fileHandle, jobStore.config)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 945, in _unpickle
8/V/jobLgOhDM        runnable = unpickler.load()
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 1749, in __new__
8/V/jobLgOhDM        return cls._resolve(*args)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 1761, in _resolve
8/V/jobLgOhDM        value = safeUnpickleFromStream(fileHandle)
8/V/jobLgOhDM      File ".../toil/src/toil/common.py", line 1347, in safeUnpickleFromStream
8/V/jobLgOhDM        return pickle.loads(string)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 1749, in __new__
8/V/jobLgOhDM        return cls._resolve(*args)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 1761, in _resolve
8/V/jobLgOhDM        value = safeUnpickleFromStream(fileHandle)
8/V/jobLgOhDM      File ".../toil/src/toil/common.py", line 1347, in safeUnpickleFromStream
8/V/jobLgOhDM        return pickle.loads(string)
8/V/jobLgOhDM      File ".../toil/src/toil/job.py", line 1841, in __setstate__
8/V/jobLgOhDM        "this job as a child/follow-on of {jobName}.".format(jobName=jobName))
8/V/jobLgOhDM    RuntimeError: This job was passed a promise that wasn't yet resolved when it ran. The job 'CWLGather' that fulfills this promise hasn't yet finished. This means that there aren't enough constraints to ensure the current job always runs after 'CWLGather'. Consider adding a follow-on indirection between this job and its parent, or adding this job as a child/follow-on of 'CWLGather'.

┆Issue is synchronized with this Jira Story ┆Issue Number: TOIL-263

[alexiswl]

Hello @mr-c, I have been able to resolve my issue. I have one step that launched some complicated outputs, I have then run an expression to take one of those files into the next step and I had a logical error in that expression that was outputting null of that expression file. I wonder if that was causing the problem?

[mr-c]

Could be! I’ll close this issue for now. I would be happy to investigate if someone can give me an example to run.