Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Action executions running out of order when delayed

See original GitHub issue

I have an Action with a concurrency policy that only allows 1 instance of the Action to run at a time. We’re intermittently seeing cases where multiple triggers are dispatched that lead to this Action being invoked close to each other. Strangely the Actions are not being invoked in the order I’d expect. I was under the impression that there’s a FIFO queue that controls how Actions are dispatched to runners that would cause the Actions to run in order? The snippet bellow shows the logs for 2 executions of this Action. In the snippet you can see that the logs indicate 587dcc8bb4eaa909d5ce0ebf was requested first, but it eventually gets delayed and 587dcc94b4eaa909d8bcc212 gets dispatched to a runner first.

This is seen in 2.1.1.

Action Execution ID: 587dcc8bb4eaa909d5ce0ebf
st2actionrunner.2517.log:2017-01-17 07:49:31,934 139902086060944 AUDIT action [-] The status of action execution is changed from requested to delayed
st2actionrunner.2517.log:2017-01-17 07:49:31,961 139902086060944 INFO action [-] The status of action execution is changed from requested to delayed
st2actionrunner.2517.log:2017-01-17 07:49:31,962 139902086060944 INFO scheduler [-] ActionExecutionScheduler is ignoring <class 'st2common
st2actionrunner.2517.log:2017-01-17 07:49:31,969 139902086059344 AUDIT action [-] Action execution requested
st2actionrunner.2520.log:2017-01-17 07:49:41,708 140505266806288 AUDIT action [-] The status of action execution is changed from requested to delayed
st2actionrunner.2520.log:2017-01-17 07:49:41,756 140505266806288 INFO action [-] The status of action execution is changed from requested to delayed
st2actionrunner.2520.log:2017-01-17 07:49:41,757 140505266806288 INFO scheduler [-] ActionExecutionScheduler is ignoring <class 'st2common
st2actionrunner.2517.log:2017-01-17 07:49:51,954 139902086060464 AUDIT action [-] The status of action execution is changed from requested to scheduled
st2actionrunner.2517.log:2017-01-17 07:49:52,035 139902086060464 INFO action [-] The status of action execution is changed from requested to scheduled
st2actionrunner.2517.log:2017-01-17 07:49:52,208 139902086061904 AUDIT worker [-] Launching action execution
st2actionrunner.2517.log:2017-01-17 07:49:52,263 139902086061904 INFO worker [-] Dispatched {~}action_execution
st2actionrunner.2517.log:2017-01-17 07:49:52,275 139902086061904 INFO base [-] Dispatching Action to a runner
st2actionrunner.2517.log:2017-01-17 07:50:00,453 139902086061904 AUDIT base [-] Liveaction completed
Action Execution ID: 587dcc94b4eaa909d8bcc212
st2actionrunner.2517.log:2017-01-17 07:49:41,413 139902089005968 AUDIT action [-] The status of action execution is changed from requested to scheduled
st2actionrunner.2517.log:2017-01-17 07:49:41,463 139902089005968 INFO action [-] The status of action execution is changed from requested to scheduled
st2actionrunner.2520.log:2017-01-17 07:49:40,838 140505265953680 AUDIT action [-] Action execution requested
st2actionrunner.2520.log:2017-01-17 07:49:41,536 140505265952400 AUDIT worker [-] Launching action execution
st2actionrunner.2520.log:2017-01-17 07:49:41,563 140505265952400 INFO worker [-] Dispatched {~}action_execution: 587dcc94b4eaa909d8bcc212 / {~}live_action: 587dcc94b4eaa909d8bcc211 with "running" status
st2actionrunner.2520.log:2017-01-17 07:49:41,571 140505265952400 INFO base [-] Dispatching Action to a runner
st2actionrunner.2520.log:2017-01-17 07:49:51,707 140505265952400 AUDIT base [-] Liveaction completed

Issue Analytics

State:
Created 7 years ago
Comments:15 (13 by maintainers)

Top GitHub Comments

1reaction

andrew-regancommented, Jun 22, 2017

@bigmstone, sorry for the delay…we are no longer seeing this issue with 2.2.1.

1reaction

bigmstonecommented, Apr 17, 2017

@andrew-regan thanks. I’m tracking this issue, so I’ll help you get it closed.

Top Results From Across the Web

time dependent workflow-action execution order

Time-dependent actions aren't executed independently. They're grouped into a single batch that starts executing within one hour after the ...

Execution order in javascript [duplicate] - delay

setTimeout is implemented in a way it's meant to execute after a minimum given delay, and once the browser's thread is free to...

Out-of-order Execution

Out-of-order execution is an approach that is used in high performance microprocessors.This approach efficiently uses instruction cycles ( is a process by which ......

Understanding Order Execution

Order execution is the process of accepting and completing a buy or sell order in the market on behalf of a client. Order...

Botched Executions | Death Penalty Information Center

Botched executions are “those involving unanticipated problems or delays that caused, at least arguably, unnecessary agony for the prisoner or that reflect ...