question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Che-machine-exec fails to execute commands after deployment is restarted

See original GitHub issue

Describe the bug

If a workspace deployment restarts for any reason, che-machine-exec will be unable to execute any tasks until the workspace is refreshed, failing with error

 Error launching task '1. Build': Request runTask failed with message: connection is closed 

refreshing the page resolves the issue. Note also that opening terminals in unaffected.

Che version

  • other: Tested on 7.26; reproduces on workspaces.openshift.com

Steps to reproduce

  1. Start a workspace from a devfile that includes commands
  2. Launch a task from the sidebar, completes successfully
  3. Delete the workspace’s pod from the commandline so that the deployment recreates it. This causes Theia to go temporarily offline as the pod is recreated
  4. Once Theia is online again, attempt to launch a task from the sidebar.

Expected behavior

Once theia reconnects, launching tasks should work

Additional context

The only suspect logs I see are in Theia:

2021-03-02 01:30:45.569 root ERROR Request runTask failed with error: connection is closed Params: Error: connection is closed
    at Object.e.create (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:801416)
    at e.remoteCall (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:803191)
    at Proxy.n.<computed>.r.charCodeAt.n.<computed> (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:803030)
    at http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:666583
    at e.<anonymous> (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:103683)
    at http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:102796
    at Object.next (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:102901)
    at http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:101813
    at new Promise (<anonymous>)
    at o (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:101558)
2021-03-02 01:30:45.579 root ERROR Error launching task 'sync-template': Request runTask failed with message: connection is closed Params: Error: Request 'run' failed
    at Proxy.<anonymous> (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:1586464)
    at t.<anonymous> (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:922871)
    at http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:922053
    at Object.next (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:922158)
    at a (http://workspace99a9e5e1468b496b-theia-3100.192.168.49.2.nip.io/theia.838437d9e5ea8a7869a6.js:1:920872)
2021-03-02 01:30:46.589 root ERROR [hosted-plugin: 48] Promise rejection not handled in one second: Error: Run task config does not return after being started , reason: Error: Run task config does not return after being started 
2021-03-02 01:30:46.589 root ERROR Promise rejection not handled in one second: Error: Run task config does not return after being started , reason: Error: Run task config does not return after being started
 
2021-03-02 01:30:46.590 root ERROR [hosted-plugin: 48] With stack trace: Error: Run task config does not return after being started
    at TasksExtImpl.<anonymous> (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:199:31)
    at step (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:48:23)
    at Object.next (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:29:53)
    at fulfilled (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:20:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) 
2021-03-02 01:30:46.590 root ERROR With stack trace: Error: Run task config does not return after being started
    at TasksExtImpl.<anonymous> (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:199:31)
    at step (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:48:23)
    at Object.next (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:29:53)
    at fulfilled (/home/theia/node_modules/@theia/plugin-ext/lib/plugin/tasks/tasks.js:20:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
RomanNikitenkocommented, Mar 3, 2021

@RomanNikitenko I’m able to open terminals without issue in my testing, it’s only devfile commands that have the issue.

OK, It’s helpful to localize the problem, looks like the problem on plugin side. thank you, Angel!

0reactions
che-botcommented, Mar 6, 2022

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to run command after initialization - kubernetes
My application requires configuration after the process has completely started. The postStart command was running before the app was ready.
Read more >
Provisioning templates failing after using reboot commands
If I remove the reboot command, the other actions work fine. Running 9.6 SP2. This is all being done in the System Config...
Read more >
How to Restart Kubernetes Pods With Kubectl - Spacelift
There is no kubectl restart [podname] command for use with Kubernetes. ... They should run until they are replaced by a new deployment....
Read more >
Kubernetes troubleshooting: 6 ways to find and fix issues
If you run kubectl get pods and see that some pods are being restarted, the next thing to do is to check why....
Read more >
Kubernetes CrashLoopBackOff Error: What It Is and How to Fix It
This error indicates that a pod failed to start, Kubernetes tried to restart it, ... Run the following command to retrieve the kubectl...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found