JupyterHub with Docker Spawner and Docker Swarm

Hello,

So I’m trying to run JupyterHub using Docker Spawner and Docker Swarm and running it locally. I’m following your workshop tutorial here : dockerspawner. I want to run it locally without using carina, in order to deploy laterly to other cloud services using Docker Swarm. So I started my own Docker Swarm as it explained here : dockerswarm.

Based own your explanation, I’ve got this configuration files:

• launch.sh

#!/bin/sh
set -euo pipefail

export DNSNAME=localhost
export JUPYTERHUB_USERS=myself
export JUPYTERHUB_ADMINS=myself

export OAUTH_CLIENT_ID=<my_oauth_client_id>
export OAUTH_CLIENT_SECRET=<my_oauth_client_secret>
#export OAUTH_CALLBACK_URL=http://localhost:8000/hub/oauth_callback

docker build -t myjupyterhub .
#docker pull jupyterhub/singleuser

docker run --detach \
  --name jupyterhub \
  -p 80:8000 \
  -p 8081:8081 \
  -e DOCKER_CERT_PATH=${DOCKER_CERT_PATH} \
  -e JUPYTERHUB_USERS=${JUPYTERHUB_USERS} \
  -e JUPYTERHUB_ADMINS=${JUPYTERHUB_USERS} \
  -e DOCKER_HOST=https://${DOCKER_HOST#tcp://} \
  -e HUB_IP_CONNECT=${DNSNAME} \
  -e OAUTH_CALLBACK_URL=http://${DNSNAME}/hub/oauth_callback \
  -e OAUTH_CLIENT_ID=${OAUTH_CLIENT_ID} \
  -e OAUTH_CLIENT_SECRET=${OAUTH_CLIENT_SECRET} \
  myjupyterhub

open http://$(docker port jupyterhub 8000)

• jupyter_config.py

import os
import re

c = get_config()

c.JupyterHub.hub_ip = "0.0.0.0"

c.JupyterHub.spawner_class = "dockerspawner.DockerSpawner"
c.DockerSpawner.tls_verify = True
c.DockerSpawner.tls_ca = "/etc/docker/ca.pem"
c.DockerSpawner.tls_cert = "/etc/docker/server-cert.pem"
c.DockerSpawner.tls_key = "/etc/docker/server-key.pem"
c.DockerSpawner.container_ip = "0.0.0.0"
c.DockerSpawner.hub_ip_connect = os.environ["HUB_IP_CONNECT"]

c.JupyterHub.authenticator_class = "oauthenticator.GoogleOAuthenticator"

c.JupyterHub.login_url = "/hub/oauth_login"

def userlist(varname):
    """
    Intercept an environment variable as a whitespace-separated list of GitHub
    usernames.
    """

    parts = re.split("\s*,\s*", os.environ[varname])
    return set([part for part in parts if len(part) > 0])

c.Authenticator.whitelist = userlist("JUPYTERHUB_USERS")
c.Authenticator.admin_users = userlist("JUPYTERHUB_ADMINS")

c.GitHubOAuthenticator.oauth_callback_url = os.environ["OAUTH_CALLBACK_URL"]
c.GitHubOAuthenticator.client_id = os.environ["OAUTH_CLIENT_ID"]
c.GitHubOAuthenticator.client_secret = os.environ["OAUTH_CLIENT_SECRET"]

• Dockerfile

FROM jupyterhub/jupyterhub

RUN pip install --upgrade pip
RUN pip install dockerspawner oauthenticator

The container with the JupyterHub and DockerSpawner is created in the swarm, it launches the JupyterHub login page but with the System User authentification, and not the Google authentification sign up as I thought. So here come my questions.

I made the changes to try to make it works locally (without carina) like here jupyterhub-tutorial. I don’t see when the jupyter_config.py is called by JupyterHub. Do I have to add this file to the Dockerfile (and in the same way, the certificate and server key) ? So the created image is the container that launches the JupyterHub with DockerSpawner.

Where do I have to call the image with the notebook I want to spawn by Docker Spawner ? I have already created another docker image with Jupyter and my notebook (the singleuser image).

[ghost]

So, after long week of search, I’m little disappointed to have no found any examples or tutorials explaining how to simply deploy jupyterhub with dockerspawner in a docker swarm locally, without using any cloud solution like Carina by Rackspace with jupiterhub-carina or OpenStack with Zonca tutorial. Nevertheless, I’ve learned so much about Docker and JupyterHub, thanks respectively to the teams developing such great tools.

So here a combined solution with all jupyterhub projects, posts and help found:

• docker-stacks
• jupyterhub-deploy-docker
• compmodels
• dockerspawner in swarm deployment
• dockerspawner error 500
• etc

So, to sum up what I’ve done until now:

1. Set up a discovery service (like Consul), create a cluster swarm, create an overlay network in the swarm and deploy the swarm as explained here: Docker overlay-networking. It’s a good start to understand how Docker Swarm and Docker Network with Swarm networking work.
2. Dockerize JupterHub like in jupyterhub-deploy-docker.
3. Customize and dockerize the SingleUser container that I want to deploy with the notebooks inside.
4. Run the JupterHub container in the Docker Swarm.

Do not mix up the standalone Docker Swarm and the Docker Engine Swarm mode. Dockerspawner runs with Docker Swarm (for the moment).

Use Docker Compose to ease the build, run and deploy process.

You can generate your own SSL certificate and key here: certificates-and-security or openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs

Here is my configuration:

• jupyterhub_config.py

c.JupyterHub.config_file = '/srv/jupyterhub/jupyterhub_config.py'
c.JupyterHub.hub_ip = 'jupyterhub'
c.JupyterHub.hub_port = 8080
c.JupyterHub.port = 443

c.JupyterHub.ssl_cert = os.environ['SSL_CERT']
c.JupyterHub.ssl_key = os.environ['SSL_KEY']
c.JupyterHub.confirm_no_ssl = True

c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
c.DockerSpawner.container_image = os.environ['DOCKER_NOTEBOOK_IMAGE']
c.DockerSpawner.remove_containers = True
c.DockerSpawner.extra_create_kwargs.update({ 'command': os.environ['DOCKER_SPAWN_CMD'] })

network_name = os.environ['DOCKER_NETWORK_NAME']
c.DockerSpawner.network_name = network_name
c.DockerSpawner.use_internal_ip = True
c.DockerSpawner.extra_host_config = { 'network_mode': network_name }
# Decapricated since Docker version 1.10
#c.DockerSpawner.extra_start_kwargs = { 'network_mode': network_name }
c.DockerSpawner.notebook_dir = os.environ['DOCKER_NOTEBOOK_DIR']
c.DockerSpawner.hub_ip_connect = 'jupyterhub'
c.DockerSpawner.container_ip = "0.0.0.0"

c.JupyterHub.authenticator_class = 'oauthenticator.GoogleOAuthenticator'
c.GoogleOAuthenticator.client_id = os.environ['OAUTH_CLIENT_ID']
c.GoogleOAuthenticator.client_secret = os.environ['OAUTH_CLIENT_SECRET']
c.GoogleOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL']
#c.GoogleOAuthenticator.hosted_domain = os.environ['HOSTED_DOMAIN']

c.Spawner.tls_assert_hostname = False
c.DockerSpawner.tls_verify = True
c.DockerSpawner.tls_ca = os.environ['DOCKER_TLS_CA']
c.DockerSpawner.tls_cert = os.environ['DOCKER_TLS_CERT']
c.DockerSpawner.tls_key = os.environ['DOCKER_TLS_KEY']

c.Authenticator.admin_users = whitelist = set()
c.Authenticator.whitelist = admin = set()

join = os.path.join
here = os.path.dirname(__file__)
with open(join(here, 'userList')) as f:
    for line in f:
        if not line:
            continue
        parts = line.split()
        name = parts[0]
        whitelist.add(name)
        if len(parts) > 1 and parts[1] == 'admin':
            admin.add(name)

• .env

# Google OAuth
OAUTH_CLIENT_ID=<my_oauth_client_id>
OAUTH_CLIENT_SECRET=<my_oauth_client_secret>
OAUTH_CALLBACK_URL=<my_oauth_callback_url>

# Name of Docker machine
DOCKER_MACHINE_NAME=jupyterhub

# Name of Docker network
DOCKER_NETWORK_NAME=jupyterhub-network

# Docker notebook image
DOCKER_NOTEBOOK_IMAGE=jupyterhub/singleuser

# Notebook directory for the single-user server
DOCKER_NOTEBOOK_DIR=/home/<user>/notebooks/

# Docker run command to use when spawning single-user containers
DOCKER_SPAWN_CMD=/srv/singleuser/singleuser.sh

# Notebook to run in for the single-user server
NOTEBOOK=<mynotebook>.ipynb

# Hosted Domain
HOSTED_DOMAIN=jupyterhub

# DNS Name or address IP
HUB_IP_CONNECT=jupyterhub

# Name of JupyterHub container data volume
DATA_VOLUME_HOST=jupyterhub-data

# Data volume container mount point
DATA_VOLUME_CONTAINER=/data

• Creating network and volume:

docker network create --driver overlay --subnet=172.10.0.0/24 jupyterhub-network

docker volume create --name jupyterhub-data

I’m now facing two problems.

• Using Google OAuth for the authentification process, dockerspawner spawn the container with the email of the user for naming the container (by replacing special characters). It seems not working with an attached volume. dockerspawner failed to spawn with the associated volume data if it contains special characters. It seems like the {username} parameter in the jupyterhub_config.py does not escape characters like @.

• After deploying the swarm with virtual machines (VirtualBox under Mac OS X), one node as a swarm manager and another as a node, I run the JupyterHub container in the swarm manager, but the dockerspawner always spawn to the swarm manager node, and not to the agent node. The network is created and the dockerspawner is configured to be able to deploy to the configured network (see above).

[minrk]

The first thing missing in the original post appears to be a small typo: the config file should be jupyterhub_config.py, not jupyter_config.py. And it must be loaded by JupyterHub. If you want to do this automatically, you can use FROM jupyterhub/jupyterhub-onbuild instead of FROM jupyterhub/jupyterhub. Or, you can add the config file manually, with ADD jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py in your Dockerfile.