Can't start a workspace when MetalLB exposes the Cluster ingresses

Hello,

I’m trying to deploy eclipse che 7.3.1 version on cluster bootstrapped with Rancher RKE. The Cluster ingress is exposed with external loadbalancer IP ( with metallb) I created offline devfile and plugin registries. Che is deployed with helm charts and takes into account customization on

• offline plugin registry (setting CHE_SIDECAR_CONTAINERS_REGISTRY_URL)
• offline devfile registry (setting CHE_DEVFILE_REGISTRY_URL, CHE_DEVFILE_IMAGES_REGISTRY_URL)
• init and unified brokers ( adding CA CERTS)
• serverStrategy: multi-host
• multiuser: false

Che starts without error and accessible throught the ingress specified in values.yml

I created a workspace with Go stack (1.12.10) with devfile :

metadata:
  name: wksp-cyjq
projects:
  - name: example
    source:
      location: >-
        https://devfile-registry-che.<fqdn>/resources/golang-example-master.zip
      type: zip
components:
  - mountSources: true
    endpoints:
      - name: 8080/tcp
        port: 8080
    memoryLimit: 512Mi
    type: dockerimage
    alias: go-cli
    image: >-
      repo.<fqdn>/docker/eclipse/che-golang-1.12:7.3.1
    env:
      - value: '/go:$(CHE_PROJECTS_ROOT)'
        name: GOPATH
      - value: /tmp/.cache
        name: GOCACHE
  - id: eclipse/che-theia/7.3.1
    type: cheEditor
  - id: ms-vscode/go/0.11.4
    type: chePlugin
apiVersion: 1.0.0
commands:
  - name: run outyet
    actions:
      - workdir: '${CHE_PROJECTS_ROOT}/src/github.com/golang/example/outyet'
        type: exec
        command: go get -d && go run main.go
        component: go-cli
  - name: stop outyet
    actions:
      - type: exec
        command: kill $(pidof go)
        component: go-cli
  - name: test outyet
    actions:
      - workdir: '${CHE_PROJECTS_ROOT}/src/github.com/golang/example/outyet'
        type: exec
        command: go test
        component: go-cli
  - name: run current file
    actions:
      - workdir: '${fileDirname}'
        type: exec
        command: 'go get -d && go run ${file}'
        component: go-cli
  - name: Debug current file
    actions:
      - referenceContent: |
          {
            "version": "0.2.0",
            "configurations": [
              {
                "name": "Debug current file",
                "type": "go",
                "request": "launch",
                "mode": "auto",
                "program": "${fileDirname}",
              },
            ]
          }
        type: vscode-launch

I started the workspace and i got those logs :

Workspace start logs :

Starting Init Plugin Broker
Cleaning /plugins dir
Unified Che Plugin Broker

Starting VS Code and Theia plugins processing
Downloading VS Code extension for plugin 'ms-vscode/go/0.11.4'
Starting Che plugins and editor processing
List of plugins and editors to install
- ms-vscode/go/0.11.4 - This extension adds rich language support for the Go language
- eclipse/che-machine-exec-plugin/7.3.1 - Che Plug-in with che-machine-exec service to provide creation terminal or tasks for Eclipse CHE workspace containers.
- eclipse/che-theia/7.3.1 - Eclipse Theia
All plugins have been successfully processed
Successfully assigned che/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz to <VM>.<fqdn>
Pulling image "eclipse/che-theia-endpoint-runtime-binary:7.3.1"
Successfully pulled image "eclipse/che-theia-endpoint-runtime-binary:7.3.1"
Created container remote-runtime-injectortsr
Started container remote-runtime-injectortsr
Pulling image "<private registry>/docker/eclipse/che-golang-1.12:7.3.1"
Successfully pulled image "<private registry>/docker/eclipse/che-golang-1.12:7.3.1"
Created container go-cli
Started container go-cli
Pulling image "<private registry>/docker/eclipse/che-machine-exec:7.3.1"
Successfully pulled image "<private registry>/docker/eclipse/che-machine-exec:7.3.1"
Created container che-machine-execfi4
Started container che-machine-execfi4
Pulling image "<private registry>/docker/eclipse/che-theia:7.3.1"
Successfully pulled image "<private registry>/docker/eclipse/che-theia:7.3.1"
Created container theia-ideeul
Started container theia-ideeul
Pulling image "<private registry>/docker/eclipse/che-remote-plugin-go-1.12.9:7.3.1"
Successfully pulled image "<private registry>/docker/eclipse/che-remote-plugin-go-1.12.9:7.3.1"
Created container vscode-godny
Started container vscode-godny
Error: Failed to run the workspace: "Server 'theia' in container 'theia-ideeul' not available."

Theia logs :

kubectl -n che logs -f workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz -c theia-ideeul

chown: /projects: Operation not permitted
root INFO Theia app listening on http://0.0.0.0:3100.
root INFO unzipping the plug-in 'eclipse_che_ports_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_ports_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_theia_containers_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_theia_factory_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_theia_factory_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_theia_ssh_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia
root INFO unzipping the plug-in 'eclipse_che_welcome_plugin.theia' to directory: /tmp/theia-unpacked/eclipse_che_welcome_plugin.theia
root INFO unzipping the plug-in 'task_plugin.theia' to directory: /tmp/theia-unpacked/task_plugin.theia
root INFO unzipping the plug-in 'theia_yeoman_plugin.theia' to directory: /tmp/theia-unpacked/theia_yeoman_plugin.theia
root INFO unzipping the VS Code extension 'vscode-git-1.3.0.1.vsix' to directory: /tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix
root ERROR Uncaught Exception:  [object Object]
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_ports_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_theia_factory_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/eclipse_che_welcome_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/task_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/theia-unpacked/theia_yeoman_plugin.theia
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix
root INFO Resolved "vscode-git-1.3.0.1.vsix" to a VS Code extension "git@1.0.0" with engines: { vscode: '^1.5.0' }
root INFO PluginTheiaDirectoryHandler: accepting plugin with path /plugins/sidecars
root ERROR Failed to load plugin dependencies from '/plugins/sidecars' path Error: ENOENT: no such file or directory, open '/plugins/sidecars/package.json'
root INFO Deploying backend plugin "@eclipse-che/ports-plugin@7.3.1" from "/tmp/theia-unpacked/eclipse_che_ports_plugin.theia/lib/ports-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/theia-containers-plugin@7.3.1" from "/tmp/theia-unpacked/eclipse_che_theia_containers_plugin.theia/lib/containers-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/theia-factory-plugin@7.3.1" from "/tmp/theia-unpacked/eclipse_che_theia_factory_plugin.theia/lib/factory-plugin.js"
root INFO Deploying backend plugin "@eclipse-che/theia-ssh-plugin@7.3.1" from "/tmp/theia-unpacked/eclipse_che_theia_ssh_plugin.theia/lib/ssh-plugin-backend.js"
root INFO Deploying backend plugin "@eclipse-che/welcome-plugin@7.3.1" from "/tmp/theia-unpacked/eclipse_che_welcome_plugin.theia/lib/welcome-plugin.js"
root INFO Deploying backend plugin "task-plugin@7.3.1" from "/tmp/theia-unpacked/task_plugin.theia/lib/task-plugin-backend.js"
root INFO Deploying backend plugin "@theia/yeoman-plugin@0.0.1-1562576566" from "/tmp/theia-unpacked/theia_yeoman_plugin.theia/lib/theia-yeoman-plugin-backend-plugin.js"
root INFO Deploying backend plugin "git@1.0.0" from "/tmp/vscode-unpacked/vscode-git-1.3.0.1.vsix/extension/out/main"

kubectl -n che logs -f workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz -c vscode-godny

Starting the deployer with the list of resolvers [ LocalDirectoryPluginDeployerResolver {},
  GithubPluginDeployerResolver { unpackedFolder: '/tmp/github-remote' },
  HttpPluginDeployerResolver { unpackedFolder: '/tmp/http-remote' },
  VsCodePluginDeployerResolver { vscodeExtensionsFolder: '/tmp/vscode-extension-marketplace' } ]
Theia Endpoint 1/pid listening on port 2503
Found the list of default plugins ID on env: undefined
Found the list of plugins ID on env: local-dir:///plugins/sidecars/ms_vscode_go_0_11_4
Found the list of default plugins ID from CLI: undefined
unzipping the VS Code extension 'ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix' to directory: /tmp/vscode-unpacked/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix
PluginTheiaDirectoryHandler: accepting plugin with path /tmp/vscode-unpacked/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix
Resolving "ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix" as a VS Code extension...
Resolved "ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix" to a VS Code extension "Go@0.11.4" with engines: { vscode: '^1.25.0' }
the accepted plugins are []
the acceptedFrontendPlugins plugins are []
the acceptedBackendPlugins plugins are [ PluginDeployerEntryImpl {
    originId: 'local-dir:///plugins/sidecars/ms_vscode_go_0_11_4',
    pluginId: 'ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix',
    map: Map { 'package.json' => [Object] },
    changes:
     [ 'PluginVsCodeFileHandler', 'PluginVsCodeDirectoryHandler' ],
    acceptedTypes: [ 1 ],
    currentPath:
     '/tmp/vscode-unpacked/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix/extension',
    initPath:
     '/plugins/sidecars/ms_vscode_go_0_11_4/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix',
    resolved: true,
    resolvedByName: 'LocalDirectoryPluginDeployerResolver' } ]
local path to deploy on remote instance [ '/tmp/vscode-unpacked/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix/extension' ]
Deploying backend plugin "Go@0.11.4" from "/tmp/vscode-unpacked/ms-vscode.go.0.11.4.thkznyafqh.Go-0.11.4.vsix/extension/out/src/goMain"

kubectl -n che get events --sort-by=‘{.lastTimestamp}’ --watch

1s          Normal    Scheduled           pod/mkdir-workspaceffdikzjktx12dbo2                   Successfully assigned che/mkdir-workspaceffdikzjktx12dbo2 to <VM-name>017.<fqdn>
1s          Normal    Pulling             pod/mkdir-workspaceffdikzjktx12dbo2                   Pulling image "centos:centos7"
1s          Normal    Pulled              pod/mkdir-workspaceffdikzjktx12dbo2                   Successfully pulled image "centos:centos7"
1s          Normal    Created             pod/mkdir-workspaceffdikzjktx12dbo2                   Created container mkdir-workspaceffdikzjktx12dbo2
1s          Normal    Started             pod/mkdir-workspaceffdikzjktx12dbo2                   Started container mkdir-workspaceffdikzjktx12dbo2
1s          Normal    Scheduled           pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Successfully assigned che/workspaceffdikzjktx12dbo2.che-plugin-broker to <VM-name>025.<fqdn>
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Pulling image "nadir88/che-init-plugin-broker:nightly"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Successfully pulled image "nadir88/che-init-plugin-broker:nightly"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Created container che-init-plugin-broker-nightly
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Started container che-init-plugin-broker-nightly
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Pulling image "nadir88/che-unified-plugin-broker:nightly"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Successfully pulled image "nadir88/che-unified-plugin-broker:nightly"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Created container che-unified-plugin-broker-nightly
2s          Normal    Started             pod/workspaceffdikzjktx12dbo2.che-plugin-broker       Started container che-unified-plugin-broker-nightly
1s          Normal    Scheduled           pod/mkdir-workspaceffdikzjktx12dbo2                   Successfully assigned che/mkdir-workspaceffdikzjktx12dbo2 to <VM-name>017.<fqdn>
1s          Normal    Pulling             pod/mkdir-workspaceffdikzjktx12dbo2                   Pulling image "centos:centos7"
1s          Normal    Pulled              pod/mkdir-workspaceffdikzjktx12dbo2                   Successfully pulled image "centos:centos7"
1s          Normal    Created             pod/mkdir-workspaceffdikzjktx12dbo2                   Created container mkdir-workspaceffdikzjktx12dbo2
1s          Normal    Started             pod/mkdir-workspaceffdikzjktx12dbo2                   Started container mkdir-workspaceffdikzjktx12dbo2
2s          Normal    CREATE              ingress/ingress2kks0ak1                               Ingress che/ingress2kks0ak1
2s          Normal    CREATE              ingress/ingress2kks0ak1                               Ingress che/ingress2kks0ak1
18s         Normal    CREATE              ingress/ingress2kks0ak1                               Ingress che/ingress2kks0ak1
2s          Normal    CREATE              ingress/ingress2kks0ak1                               Ingress che/ingress2kks0ak1
0s          Normal    CREATE              ingress/ingress2kks0ak1                               Ingress che/ingress2kks0ak1
..
1s          Normal    UPDATE              ingress/ingress57febwxx                               Ingress che/ingress57febwxx
2s          Normal    UPDATE              ingress/ingress6qg1dd74                               Ingress che/ingress6qg1dd74
1s          Normal    ScalingReplicaSet   deployment/workspaceffdikzjktx12dbo2.go-cli           Scaled up replica set workspaceffdikzjktx12dbo2.go-cli-96d848845 to 1
1s          Normal    SuccessfulCreate    replicaset/workspaceffdikzjktx12dbo2.go-cli-96d848845   Created pod: workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz
1s          Normal    Scheduled           pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully assigned che/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz to <VM-name>025.<fqdn>
2s          Normal    UPDATE              ingress/ingressr5tdm92r                                 Ingress che/ingressr5tdm92r
2s          Normal    UPDATE              ingress/ingressr5tdm92r                                 Ingress che/ingressr5tdm92r
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Pulling image "eclipse/che-theia-endpoint-runtime-binary:7.3.1"
3s          Normal    UPDATE              ingress/ingressrxa7sfiy                                 Ingress che/ingressrxa7sfiy
3s          Normal    UPDATE              ingress/ingressrxa7sfiy                                 Ingress che/ingressrxa7sfiy
3s          Normal    UPDATE              ingress/ingress57febwxx                                 Ingress che/ingress57febwxx
3s          Normal    UPDATE              ingress/ingress57febwxx                                 Ingress che/ingress57febwxx
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully pulled image "eclipse/che-theia-endpoint-runtime-binary:7.3.1"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Created container remote-runtime-injectortsr
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Started container remote-runtime-injectortsr
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Pulling image "<private registry>/docker/eclipse/che-golang-1.12:7.3.1"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully pulled image "<private registry>/docker/eclipse/che-golang-1.12:7.3.1"
2s          Normal    Created             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Created container go-cli
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Started container go-cli
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Pulling image "<private registry>/docker/eclipse/che-machine-exec:7.3.1"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully pulled image "<private registry>/docker/eclipse/che-machine-exec:7.3.1"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Created container che-machine-execfi4
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Started container che-machine-execfi4
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Pulling image "<private registry>/docker/eclipse/che-theia:7.3.1"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully pulled image "<private registry>/docker/eclipse/che-theia:7.3.1"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Created container theia-ideeul
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Started container theia-ideeul
1s          Normal    Pulling             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Pulling image "<private registry>/docker/eclipse/che-remote-plugin-go-1.12.9:7.3.1"
1s          Normal    Pulled              pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Successfully pulled image "<private registry>/docker/eclipse/che-remote-plugin-go-1.12.9:7.3.1"
1s          Normal    Created             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Created container vscode-godny
1s          Normal    Started             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Started container vscode-godny
1s          Normal    DELETE              ingress/ingress2kks0ak1                                 Ingress che/ingress2kks0ak1
1s          Normal    DELETE              ingress/ingress2kks0ak1                                 Ingress che/ingress2kks0ak1
1s          Normal    DELETE              ingress/ingress2kks0ak1                                 Ingress che/ingress2kks0ak1
2s          Normal    DELETE              ingress/ingress2kks0ak1                                 Ingress che/ingress2kks0ak1
18s         Normal    DELETE              ingress/ingress2kks0ak1                                 Ingress che/ingress2kks0ak1

...

18s         Normal    DELETE              ingress/ingressxre29ylz                                 Ingress che/ingressxre29ylz
1s          Normal    ScalingReplicaSet   deployment/workspaceffdikzjktx12dbo2.go-cli             Scaled down replica set workspaceffdikzjktx12dbo2.go-cli-96d848845 to 0
1s          Normal    Killing             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Stopping container vscode-godny
1s          Normal    SuccessfulDelete    replicaset/workspaceffdikzjktx12dbo2.go-cli-96d848845   Deleted pod: workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz
1s          Normal    Killing             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Stopping container theia-ideeul
1s          Normal    Killing             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Stopping container che-machine-execfi4
1s          Normal    Killing             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Stopping container go-cli
1s          Normal    Killing             pod/workspaceffdikzjktx12dbo2.go-cli-96d848845-fcbqz    Stopping container go-cli

[NM4]

Hello, Any update please on this issue ? As summary : I deployed che with chectl on k8s platform bootstrapped with Rancher RKE and the ingress-controller is exposed with metallb loadbalancer.

When I create a workspace with this Devfiles :

metadata:
  name: wksp-test
components:
  - id: ws-skeleton/eclipseide/latest 
    type: cheEditor
apiVersion: 1.0.0

I get the Eclipse Ide interface.

But with

metadata:
  name: wksp-test
components:
  - id: eclipse/che-theia/7.3.1
    type: cheEditor
apiVersion: 1.0.0

I get

Error: Failed to run the workspace: "Server 'theia' in container 'theia-ideaff' not available."

with ep created :

kubectl -n che get ep
NAME                                 ENDPOINTS                                                       AGE
che-host                             10.42.8.3:8080,10.42.8.3:8087                                   49m
devfile-registry                     10.42.8.2:8080                                                  49m
plugin-registry                      10.42.6.169:8080                                                49m
server0blqlr97-theia-idek1f          10.42.8.13:13132,10.42.8.13:13131,10.42.8.13:3130 + 2 more...   92s
servertog353f2-che-machine-execcws   10.42.8.13:4444                                                 92s

and che-server logs :

2019-11-05 13:31:22,704[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 678] - Creating deployment ‘workspaceufop0p3tkbh8siy5.maven’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,705[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 698] - Creating machine ‘maven’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,706[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 698] - Creating machine ‘che-machine-execcws’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,708[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 698] - Creating machine ‘theia-idek1f’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,710[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 698] - Creating machine ‘vscode-javaldv’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,711[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 684] - Pods creation finished in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:22,962[aceSharedPool-9] [DEBUG] [.i.k.KubernetesInternalRuntime 329] - Waiting to start machines of workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,371[neSharedPool-13] [DEBUG] [.i.k.KubernetesInternalRuntime 395] - Performing servers check for machine ‘che-machine-execcws’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,371[neSharedPool-15] [DEBUG] [.i.k.KubernetesInternalRuntime 395] - Performing servers check for machine ‘theia-idek1f’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,373[neSharedPool-13] [DEBUG] [.i.k.KubernetesInternalRuntime 404] - Servers checks done for machine ‘che-machine-execcws’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,377[neSharedPool-14] [DEBUG] [.i.k.KubernetesInternalRuntime 395] - Performing servers check for machine ‘vscode-javaldv’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,377[neSharedPool-16] [DEBUG] [.i.k.KubernetesInternalRuntime 395] - Performing servers check for machine ‘maven’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,377[neSharedPool-16] [DEBUG] [.i.k.KubernetesInternalRuntime 404] - Servers checks done for machine ‘maven’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:31:31,378[neSharedPool-14] [DEBUG] [.i.k.KubernetesInternalRuntime 404] - Servers checks done for machine ‘vscode-javaldv’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:34:31,922[aceSharedPool-9] [WARN ] [.i.k.KubernetesInternalRuntime 241] - Failed to start Kubernetes runtime of workspace workspaceufop0p3tkbh8siy5. Cause: Server ‘theia’ in container ‘theia-idek1f’ not available. 2019-11-05 13:34:31,924[ServersChecker] [DEBUG] [.i.k.KubernetesInternalRuntime 404] - Servers checks done for machine ‘theia-idek1f’ in workspace ‘workspaceufop0p3tkbh8siy5’ 2019-11-05 13:34:32,544[aceSharedPool-9] [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 854] - Workspace ‘che:wksp-v9h3’ with id ‘workspaceufop0p3tkbh8siy5’ start failed

[mmorhun]

@NM4 could you please try to start a workspace without remote plugins? For example (the simplest case):

metadata:
  name: wksp-test
components:
  - id: eclipse/che-theia/7.3.1
    type: cheEditor
apiVersion: 1.0.0