I cannot create a fully functional CHE workspace using Docker beta for mac

The issue

The configuration is illustrated in the following diagram :

Both Che server and its workspaces are running in a Docker container. Both container are running in a Mac xhyve VM.

With Docker for Mac beta, wsagent URL is localhost (URL1) when connecting from outside the VM (i.e. the browser) whereas wsmaster need to use the VM IP address to connect to the wsagent (URL2).

However I could not manage to configure two distinct values for URL1 and URL2. Both are set through the same property: machine.docker.local_node_host.

How to reproduce the issue

1. Install Docker for Mac Beta

2. Retrieve the IP of the xhyve VM. To get a shell into the VM screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty and to retrieve the IP ip a show eth0 | grep 'inet ' | cut -d/ -f1 | awk '{ print $2}'

3. Create che.properties file:

   export DOCKER_VM_IP=<put the IP here>
   echo "machine.docker.local_node_host=$DOCKER_VM_IP" > che.properties
   

4. Run docker from the che.properties folder to start a shell in a new Che container:

   docker run --name=che \
          -v /home/user/che/lib:/home/user/che/lib-copy \
          -v /home/user/che/workspaces:/home/user/che/workspaces \
          -v /home/user/che/storage:/home/user/che/storage \
          -v /var/run/docker.sock:/var/run/docker.sock \
          -v $(pwd)/che.properties:/home/user/.che/che.properties \
          -p 8080:8080 \
          -ti \
          --entrypoint=bash \
          codenvy/che:nightly
   # Retrieve the container IP address and add a property to che.properties
   export CHE_SERVER_IP=$(ip address show eth0 | grep 'inet ' |  cut -d/  -f1 | awk '{ print $2}')
   echo "machine.docker.che_api.endpoint=http://${CHE_SERVER_IP}:8080/wsmaster/api" >> /home/user/.che/che.properties
   # Add user to the group with gid=50 (that's the VM docker group)
   sudo usermod -aG 50 user
   sudo su -l user
   export JAVA_HOME=/opt/jre1.8.0_65
   export PATH=$JAVA_HOME/bin:$PATH
   export CHE_LOCAL_CONF_DIR=/home/user/.che
   # Run che
   /home/user/che/bin/che.sh -c run
   

5. Use a browser to open http://localhost:8080 and try to create a new project. The websocket should fail to connect to the wsagent because it’s using the VM internal IP instead of localhost:

6. Set machine.docker.local_node_host=localhost in che.properties and restart the che server. This time the che server fails to connect to the wsagent:

016-06-10 15:56:59,252[kspaceManager-0]  [ERROR] [.c.a.m.s.w.WsAgentLauncherImpl 103]  - Fail pinging ws agent. Workspace ID:workspace7p1virvy0wws6pno. Url:http://localhost:33041/wsagent/ext/. Timestamp:{}

Workaround

We can add an alias for the loopback interface in Mac OSX to forward every $DOCKER_VM_IP connection to localhost:

1. sudo ifconfig lo0 alias $DOCKER_VM_IP
2. revert machine.docker.local_node_host in che.properties: machine.docker.local_node_host=$DOCKER_VM_IP
3. Restart Che and verify that it’s possible to create a new project.

[TylerJewell]

@LeDominik - Thanks for writing in. Let’s see if we can get you past these problems.

First, we will very soon be releasing a wrapper container that will eliminate the need for doing some of these complicated mounts. The run syntax will essentially just be “docker run --rm codenvy/che start”. And this container will take care to actually launch the Che server container with all of the appropriate configuration. These sorts of volume mounting issues should disappear.

Second, can you paste the full syntax of what you used to launch the Che container?

Third, here is what is happening when you get that error message. When the Che container starts, we have to move files from inside the container onto your host system. So the sequence goes like:

1. Inside the container, we have a /lib-copy directory.
2. When the container boots, we copy stuff from /lib into /lib-copy. We cannot have it in /lib-copy to begin with because /lib-copy is hosted mounted to a directory that doesn’t exist, which will just have the unfortunate affect of wiping out /lib-copy in the container. So, we have to do the mount first, and then second copy the files into /lib-copy. Those files then get placed into the host at /home/user/che/lib/.
3. We need the ability to write files into that host directory. If we cannot, then when we perform the copy of the files inside the container, they will not make it onto the mount.
4. When Che is running and you create a workspace, that workspace is in another container. That workspace container needs the files that was in /lib-copy. So in that container we now mount / home/user/che/lib/, and this is what allows the workspace container to discover the ws-agent.zip.

If that particular directory is unmountable, there is a way to configure che.properties to tell this to go into another directory. If that is the issue, I think we can walk you through it. Or, you can make that directory editable and usable by Che 😃

[l0rd]

@TylerJewell Copy/pasting the following commands should do the trick 😉 (works only if the PR #1576 is merged)

export DOCKER_VM_IP=$(docker run --rm --net host alpine sh -c "ip a show eth0" | grep 'inet ' | cut -d/ -f1 | awk '{ print $2}')
sudo ifconfig lo0 alias $DOCKER_VM_IP
docker run --name=che \
    -v /home/user/che/lib:/home/user/che/lib-copy \
    -v /home/user/che/workspaces:/home/user/che/workspaces \
    -v /home/user/che/storage:/home/user/che/storage \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 8080:8080 \
    codenvy/che:nightly --remote:$DOCKER_VM_IP