Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
If async storage pod gets evicted, workspaces cannot be stopped or restarted.
See original GitHub issueEdit: Looking at what limited information I can get from Hosted Che, this may be semi-unrelated to async-storage. The container in the stuck terminating workspace pod that won’t shut down is the che-docs antora container, which was running a gulp task. The rsync pod seems to have terminated successfully.
Describe the bug
While using a workspace with asynchronous storage enabled on Hosted Che, the async storage pod was evicted from the cluster. After this happened, the pod was not re-created and workspace stop hangs. Eventually it seems like stop times out, and I get a red icon suggesting stop failed in some way, with the message ERROR on hover. Afterwards
-
Restarting the workspace is not possible; immediately after metadata broker completes, the message
Error: Failed to run the workspace: “Unable to start the workspace ‘workspacewcbpmj0robtm3x3a’ due to an internal inconsistency while composing the workspace runtime.Please report a bug. If possible, include the details from Che devfile and server log in bug report (your admin can help with that)”
is logged
-
The workspace pod is stuck in a terminating state (presumably attempting to sync with the rsync server that was evicted).
-
Async pod was evicted with reason
The node was low on resource: ephemeral-storage. Container async-storagev5xh9xb3 was using 276Ki, which exceeds its request of 0.
Before workspace was stopped, Theia showed a tooltip stating that git checkout failed due to quota, but I wasn’t able to get more information on this.
Che version
- latest
Steps to reproduce
- Create workspace with async storage enabled
- Force eviction of async pod (somehow?)
- Try to stop workspace
Runtime
- Openshift
v3.11.82
Environment details
- Namespace:
amisevsk-cheon Hosted Che to see terminating pods and evicted async storage
Issue Analytics
- State:
- Created 3 years ago
- Comments:6 (6 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
It will be on prod only with 7.19.x version
I do suspect that the issue should be fixed with https://github.com/eclipse/che/issues/17616