question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Mistral workflow stuck in RUNNING state after error encountered.

See original GitHub issue

A task was marked as RUNNING despite extra information being available indicating an error occurred.

  "elapsed_seconds": 231993.945685,
  "web_url": "https://automate.dm.gg/#/history/58b06299fd6f06179d18eb94/general",
  "parent": "58b05f28fd6f0640445ed4d5",
  "result": {
    "tasks": [
      {
        "state_info": null,
        "name": "check_host_health",
        "created_at": "2017-02-24 16:43:05.537055",
        "updated_at": null,
        "id": "0986cc88-4696-4890-8b74-0cc9c0b552b9",
        "workflow_execution_id": "1417305b-288d-40f7-b7fc-324ba897fd3a",
        "state": "RUNNING",
        "result": [],
        "published": {},
        "input": null,
        "workflow_name": "st2dm_upgrade_pkg.post_upgrade_pkg.check_environment"
      }
    ],
    "extra": {
      "state_info": "list index out of range",
      "state": "ERROR"
    }

The task that was called is shown below and the error is referring to mistral being unable to publish the variable because YAQL detected the list index out of range.:

      check_host_health:
        action: livestatus.get
        input:
          table: services
          columns: [ host_address ]
          stats:
            - state != 0
            - state_type = 1
            - "&2"
          filters:
            - host_address ~ ^<% $.host %>
            - display_name ~ OS::Hardware
            - "!"
            - display_name ~ SSH::OOB service status
            - "!"
            - display_name ~ HTTP::Zend OPCache
            - "!"
        publish:
            health_status: <% int(task(check_host_health).result.result[0][1]) %>
        on-success:
          - remove_schedule_downtime: <% $.health_status = 0 %>
          - abort_upgrade: <% $.health_status != 0 %>
        on-error: abort_upgrade

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:15 (15 by maintainers)

github_iconTop GitHub Comments

1reaction
nzloshcommented, Mar 2, 2017

@emptywee are the errors you posted related to list index out of range or are they all errors related to tasks remaining in RUNNING state?

@lakshmi-kannan the use case posted in gist isn’t quite the same as the one I posted. Here is a simple workflow to reproduce the error

actions/yaql_out_of_range.yaml

---
name: yaql_out_of_range
pack: test
description: "Provoke a list index out of range error"
runner_type: mistral-v2
entry_point: workflow/yaql_out_of_range.yaml
enabled: True
parameters:
    list_param:
        type: array
        default: []
        description: An empty list.

actions/workflow/yaql_out_of_range.yaml

version: '2.0'

test.yaql_out_of_range:
    description: test yaql list index out of range.
    type: direct
    input:
        - list_param
    tasks:
        read_consul:
            action: core.local
            input:
                cmd: "echo <% list_param %>"
            publish:
                bad_yaql_ref: <% $.list_param[0][1] %>
            on-success: good
            on-error: bad

        good:
            action: core.noop

        bad:
            action: core.noop
0reactions
LindsayHillcommented, Jun 19, 2018

Mistral callbacks, etc have changed a lot in the last 12 months. Going to close this since there’s no current, consistent way of reproducing it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to troubleshoot a mistral workflow that is stuck in "running ...
Is there an easy way to determine why a mistral workflow still sits on Running even though the last action that ran within...
Read more >
Bug #1756353 “Mistral workflow stuck in RUNNING state after ...
A task was marked as RUNNING despite the error reported in mistral engine and executor logs: 2018-03-16 14:01:54.000 5715 INFO ...
Read more >
Mistral workflow stuck in RUNNING state after error encountered. -
A task was marked as RUNNING despite extra information being available indicating an error occurred. ``` "id": "58b06299fd6f06179d18eb94",
Read more >
1703618 – ceph_install mistral task stuck in state RUNNING
WORKAROUND: Identify the stuck task's action id and manually set it state to SUCCESS. To identify the stuck task's action id, use a...
Read more >
Mistral basic example never completes? - Google Groups
1) Does it get stuck in the running state? ... To do so, get Mistral workflow-execution id st2 execution get ... ERROR: 400...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found