Failed to run the workspace: mkdir /plugins/sidecars: permission denied

Describe the bug

I am trying to get Che 7.0.0 multiuser running on K8’s, deployed via Rancher using the Helm charts in the repo. I’ve managed to get everything running now so far as the Che dashboard appears. I am however now stuck from going any further as, when starting a workspace I get the following error:

Error: Failed to run the workspace: "Plugins installation process failed. Error: Plugin broking process for workspace workspacehe0g3fhudssx2s7n failed with error: mkdir /plugins/sidecars: permission denied"

The first thing I did was to check the physical volume, it is just a local folder path on the node at the moment for testing purposes. Here is it’s contents:

root@staging1:/test15# ls -la
total 12
drwxrwxrwx  3 root root 4096 Aug 24 11:47 .
drwxr-xr-x 25 root root 4096 Aug 24 11:47 ..
drwxr-xr-x  4 1724 root 4096 Aug 24 11:47 workspacehe0g3fhudssx2s7n

So it looks like Che has been able to create the workspace’s folder in the volume without any permission errors. It has also created the plugins folder:

root@staging1:/test15/workspacehe0g3fhudssx2s7n# ls -la
total 16
drwxr-xr-x 4 1724 root 4096 Aug 24 11:55 .
drwxrwxrwx 3 root root 4096 Aug 24 11:55 ..
drwxrwxrwx 2 1724 root 4096 Aug 24 11:55 che-logs-che-plugin-broker
drwxrwxrwx 2 1724 root 4096 Aug 24 11:55 plugins

So…I’m at a loss as to why the plugins/sidecars can’t be create. I’ve tried deleting all folders and starting again but end up with the same problem.

I’d really appreciate any help or pointers on this, it’s stopped us moving to the new Che for quite a while now and I’m at a loss.

Thanks!

[davidwindell]

Can confirm, the problem is isolated to hostPath volumes (which I was using for testing).

Switching to EBS finally solves this issue for me. Thank you @amisevsk for your help with this.

[davidwindell]

Thanks @amisevsk, I’m going to try and use an EBS volume and will report back in a few days.