Trying to get this working to for deploying JupyterLab and using latest 0.7b1 of JupyterHub

Perhaps this should be added as an issue on the JupyterHub repo … but it also feels appropriate to post here, given that this deployment is intended to be a best-practice approach to combining various repos … like JupyterHub, JupyterLab and dockerspawner.

With the initial release by @minrk of the latest version of Jupyterhub, I’m trying to leverage this deployment to see how JupyterLab might work against this preliminary release.

It seems that JupyterLab should work out of the box by extending one of the docker-stack images, as per below, but I’m struggling a bit to get things working … and would appreciate being pointed in the right direction to help resolve.

# replace with your desired base stack
FROM jupyter/scipy-notebook
RUN conda install jupyterlab
CMD ["start.sh", "jupyter", "lab"]

My use-case is seeking to extend the datascience-notebook, such that I can get it working in conjunction with JupyterLab. My image to get deployed on JupyterHub looks like this … I’m not sure those additional parameters in CMD are really necessary, but through trial and error … they got me closer to a working solution … and were based on some of the related issues I saw in the dockerspawner repo here and here.

FROM jupyter/datascience-notebook
RUN conda config --add channels conda-forge
RUN conda install -c conda-forge jupyterlab --yes
RUN jupyter serverextension enable --py --sys-prefix jupyterlab
CMD ["start.sh", "jupyter", "lab", "--SingleUserNotebookApp.default_url=/lab", "--SingleUserNotebookApp.disable_user_config=True"]

My Dockerhub dockerfile looks like this … and it based on the on that one @parente had ready to roll here on the 0.7 release … but I don’t know whether it’s missing something to make it work with the 0.7.0b1 image generated and on docker registry?

FROM jupyterhub/jupyterhub-onbuild:0.7.0b1

# Install dockerspawner and its dependencies
RUN /opt/conda/bin/pip install \
    oauthenticator==0.5.* \
    dockerspawner==0.5.*

# install docker on the jupyterhub container
RUN wget https://get.docker.com -q -O /tmp/getdocker && \
    chmod +x /tmp/getdocker && \
    sh /tmp/getdocker

# Copy TLS certificate and key
ENV SSL_CERT /srv/jupyterhub/secrets/jupyterhub.crt
ENV SSL_KEY /srv/jupyterhub/secrets/jupyterhub.key
COPY ./secrets/*.crt $SSL_CERT
COPY ./secrets/*.key $SSL_KEY
RUN chmod 700 /srv/jupyterhub/secrets && \
    chmod 600 /srv/jupyterhub/secrets/*

COPY ./userlist /srv/jupyterhub/userlist

My .env and jupyterhub_config.py files are pretty much the same as already in this repo, but I was playing around with the following additional lines … commenting/uncommenting to try to get things to work. Given that my image above ends in CMD ["start.sh", "jupyter", "lab"] … I wasn’t sure if that was enough … or if I needed another explicit DOCKER_SPAWN_CMD in the ‘.env’ file… it seemed it didn’t like anytime I tried to use the extra_create_kwargs variant, especially if there are spaces in the command.

#spawn_cmd = os.environ.get('DOCKER_SPAWN_CMD', "start-singleuser.sh")
#c.DockerSpawner.extra_create_kwargs.update({ 'command': spawn_cmd })
#c.DockerSpawner.extra_create_kwargs.update({'command': 'start.sh jupyter lab'})

It also wasn’t clear to be which variant of adding ‘/lab’ to default to JupyterLab was going to work … since I this issue seems to suggest that Dockerspawner doesn’t have the ability to set a default_url … which is why I was playing around with mixing in plain Spawner and SingleUserNotebookApp variants … but it’s not clear to me whether you can mix/match these.

#c.Spawner.default_url='/user/%U/lab'
c.Spawner.default_url = '/lab'
#c.SingleUserNotebookApp.default_url= '/user/%U/lab'
#c.SingleUserNotebookApp.disable_user_config = True

The only other variant is that I’m using a GitLabAuthenticatior (versus Github in here)… but that all seems to work OK.

When I run up the hub and start my server … I end up with this … at the endpoint of https://myserver/user/testuser. In the docker-compose file, I had added ports “80:8080” … just so I didn’t need to add a port to the ‘myserver’ above … but don’t think that should be the source of the problem.

I can see that a new container was spawned OK … and I know that running up the image I build … exposing port 8888 and navigating to that in a browser works … but somehow adding it into the jupyterhub mix … something is failing.

CONTAINER ID        IMAGE                                                         COMMAND                  CREATED             STATUS              PORTS                                                  NAMES
dbafa6cc1c6e        jupyter/datascience-notebook:jlab0.1   "tini -- start.sh jup"   33 seconds ago      Up 33 seconds       8888/tcp                                               jupyter-testuser
2c44acdb448d        jupyterhub                                                    "jupyterhub -f /srv/j"   2 minutes ago       Up About a minute   0.0.0.0:443->443/tcp, 8000/tcp, 0.0.0.0:80->8080/tcp   jupyterhub

I don’t know enough about the use of a proxy here … and whether that is the failing part. Here are the logs. Is there anything glaring error in there that looks obvious? Should I have done something more to ‘install’ Jupyterlab on the image … or was that conda command enough? How should I be setting up the default_url to take me to a working ‘lab’ session?

jupyterhub-deploy-docker$ docker-compose up
Starting jupyterhub
Attaching to jupyterhub
jupyterhub | [I 2016-11-15 13:55:26.486 JupyterHub app:724] Loading cookie_secret from /data/jupyterhub_cookie_secret
jupyterhub | [W 2016-11-15 13:55:26.527 JupyterHub app:365] 
jupyterhub |     Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require restarting the proxy.
jupyterhub |     Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config to avoid this message.
jupyterhub |     
jupyterhub | [I 2016-11-15 13:55:26.564 JupyterHub dockerspawner:332] Container 'jupyter-testuser' is gone
jupyterhub | [W 2016-11-15 13:55:26.565 JupyterHub dockerspawner:303] container not found
jupyterhub | [I 2016-11-15 13:55:26.589 JupyterHub app:1450] Hub API listening on http://jupyterhub:8080/hub/
jupyterhub | [I 2016-11-15 13:55:26.592 JupyterHub app:1173] Starting proxy @ http://*:443/
jupyterhub | 13:55:26.788 - info: [ConfigProxy] Proxying https://*:443 to http://jupyterhub:8080
jupyterhub | 13:55:26.791 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
jupyterhub | [I 2016-11-15 13:55:26.799 JupyterHub app:1482] JupyterHub is now running at http://127.0.0.1:443/
^[[1;7Cjupyterhub | [I 2016-11-15 13:55:47.987 JupyterHub log:100] 302 GET / (@172.18.0.2) 1.47ms
jupyterhub | [I 2016-11-15 13:55:48.008 JupyterHub log:100] 302 GET /hub (@172.18.0.2) 1.77ms
jupyterhub | [W 2016-11-15 13:55:48.025 JupyterHub pages:32] Disallowing redirect outside JupyterHub: ''
jupyterhub | [I 2016-11-15 13:55:48.029 JupyterHub log:100] 302 GET /hub/ (@172.18.0.2) 1.27ms
jupyterhub | [I 2016-11-15 13:55:48.046 JupyterHub log:100] 302 GET /oauth_login (@172.18.0.2) 0.79ms
jupyterhub | [I 2016-11-15 13:55:48.061 JupyterHub oauth2:37] oauth redirect: 'https://myserver.com/hub/oauth_callback'
jupyterhub | [I 2016-11-15 13:55:48.064 JupyterHub log:100] 302 GET /hub/oauth_login (@172.18.0.2) 1.34ms
jupyterhub | http://myserver.com:10080//oauth/token?grant_type=authorization_code&code=f96f907debf86af5e78682dbe0ba4d608dddf7f1c7a3cb4c635e9ba329a47111&redirect_uri=https%3A%2F%2Fmyserver.com%2Fhub%2Foauth_callback&client_id=75c36207f96584c67123ea203118fe353b2be4dc783b6da2d012e1f4acf84457&client_secret=d215282cb8163d5a313493ae953e771ee2cb482e3e833e6752c566d3dc5eddf3
jupyterhub | [I 2016-11-15 13:55:55.715 JupyterHub log:100] 302 GET /hub/oauth_callback?code=f96f907debf86af5e78682dbe0ba4d608dddf7f1c7a3cb4c635e9ba329a47111 (@172.18.0.2) 233.60ms
jupyterhub | [I 2016-11-15 13:55:55.770 JupyterHub log:100] 200 GET /hub/home (testuser@172.18.0.2) 32.03ms
jupyterhub | [I 2016-11-15 13:55:58.033 JupyterHub log:100] 302 GET /hub/spawn (testuser@172.18.0.2) 3.79ms
jupyterhub | [I 2016-11-15 13:55:58.048 JupyterHub log:100] 302 GET /user/testuser (@172.18.0.2) 0.84ms
jupyterhub | [I 2016-11-15 13:55:58.068 JupyterHub dockerspawner:332] Container 'jupyter-testuser' is gone
jupyterhub | [W 2016-11-15 13:55:58.068 JupyterHub dockerspawner:303] container not found
jupyterhub | [I 2016-11-15 13:55:58.092 JupyterHub dockerspawner:332] Container 'jupyter-testuser' is gone
jupyterhub | [I 2016-11-15 13:55:58.142 JupyterHub dockerspawner:394] Created container 'jupyter-test-user' (id: dbafa6c) from image jupyter/datascience-notebook:jlab0.1
jupyterhub | [I 2016-11-15 13:55:58.142 JupyterHub dockerspawner:404] Starting container 'jupyter-testuser' (id: dbafa6c)
jupyterhub | [I 2016-11-15 13:55:59.688 JupyterHub base:322] User testuser server took 1.619 seconds to start
jupyterhub | [I 2016-11-15 13:55:59.688 JupyterHub orm:188] Adding user testuser to proxy /user/testuser => http://172.18.0.3:8888
jupyterhub | [I 2016-11-15 13:55:59.704 JupyterHub log:100] 302 GET /hub/user/testuser (testuser@172.18.0.2) 1638.97ms
jupyterhub | [I 2016-11-15 13:55:59.850 JupyterHub log:100] 302 GET /static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=9b2c8d3489227115310662a343fce11c (@172.18.0.2) 0.95ms
jupyterhub | [I 2016-11-15 13:55:59.855 JupyterHub log:100] 302 GET /static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=7afb461de36accb1aa133a1710f5bc56 (@172.18.0.2) 0.78ms
jupyterhub | [I 2016-11-15 13:55:59.871 JupyterHub log:100] 302 GET /static/style/style.min.css?v=f6c09475baf6beabd41f8fe518601204 (@172.18.0.2) 1.36ms
jupyterhub | [I 2016-11-15 13:55:59.875 JupyterHub log:100] 302 GET /custom/custom.css (@172.18.0.2) 0.77ms
jupyterhub | [W 2016-11-15 13:55:59.886 JupyterHub log:100] 404 GET /hub/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=7afb461de36accb1aa133a1710f5bc56 (@172.18.0.2) 1.05ms
jupyterhub | [W 2016-11-15 13:55:59.892 JupyterHub log:100] 404 GET /hub/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=9b2c8d3489227115310662a343fce11c (@172.18.0.2) 0.68ms
jupyterhub | [I 2016-11-15 13:55:59.901 JupyterHub log:100] 302 GET /static/components/es6-promise/promise.min.js?v=f004a16cb856e0ff11781d01ec5ca8fe (@172.18.0.2) 1.02ms
jupyterhub | [I 2016-11-15 13:55:59.906 JupyterHub log:100] 302 GET /static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce (@172.18.0.2) 0.78ms
jupyterhub | [W 2016-11-15 13:55:59.926 JupyterHub log:100] 404 GET /hub/custom/custom.css (testuser@172.18.0.2) 13.59ms
jupyterhub | [W 2016-11-15 13:55:59.928 JupyterHub log:100] 404 GET /hub/static/components/es6-promise/promise.min.js?v=f004a16cb856e0ff11781d01ec5ca8fe (@172.18.0.2) 0.74ms
jupyterhub | [W 2016-11-15 13:55:59.932 JupyterHub log:100] 404 GET /hub/static/style/style.min.css?v=f6c09475baf6beabd41f8fe518601204 (@172.18.0.2) 0.67ms
jupyterhub | [I 2016-11-15 13:55:59.991 JupyterHub log:100] 302 GET /static/base/images/logo.png?v=7c4597ba713d804995e8f8dad448a397 (@172.18.0.2) 0.96ms
jupyterhub | [W 2016-11-15 13:56:00.008 JupyterHub log:100] 404 GET /hub/static/base/images/logo.png?v=7c4597ba713d804995e8f8dad448a397 (@172.18.0.2) 0.82ms
^[[1;7C^[[1;7C

Thanks alot. Colum

[erik-stephens]

The entry for my .env file looks like this: DOCKER_SPAWN_CMD=“/usr/local/bin/start-singleuser.sh --SingleUserNotebookApp.default_url=/lab” … I was sometimes leaving out the /usr/local/bin part … but from your entry above … perhaps I shouldn’t be containing this command in quotation marks at all. I’ll see if that makes a difference.

I’m pretty sure docker treats everything after the = literally, so you don’t quote your values.

[Analect]

@jtyberg Ref. jupyterlab install on stack image … sorry, for my wayward comment above … of course that’s already what we are doing! … getting mixed up here … and so probably an opportune time to close this verbose issue. Your help is much appreciated with this stuff.