Installing https support via Let's Encrypt appears broken (instructions problematic)

On a freshly installed jupyterhub that is visible to the outside world, I followed the Let’s Encrypt instructions on the Enabling HTTPS document page. I confirmed sudo -E tljh-config show returns the expected content compared to what is in the documentation.

Problem 1) When I do sudo -E tljh-config reload proxy, nothing happens. In fact, I realized that the connection hangs if you are doing this through the terminal on the jupyterhub. This is not surprising since it is shutting down http and turning on https. However, there is no warning in the documentation that this will happen.

Problem 2) When I try to go to the https connection, it is active, but the certificate is NOT being recognized as “verified by a third party.” (in Chrome, this is NET::ERR_CERT_AUTHORITY_INVALID) It does appear to be created since its name is “TRAEFIK DEFAULT CERT”.

The documentation should be updated to fix Problem 1, and I would appreciate any hints as to how to ‘redo’ the proxy connection properly. I did try re-running sudo -E tljh-config reload proxy from ssh, and it returned Proxy reload with new configuration complete but didn’t fix the issue.

We did revert to a snapshot of the VM from before activation of HTTPS and try the instructions from an SSH terminal. The result was the same except that sudo -E tljh-config reload proxy from ssh, returned Proxy reload with new configuration complete (since the http session terminal was not used), but the certificate is still not recognized as a third party verified certificate. Is there something more we need to do?

[dschofield]

error msg="Unable to obtain ACME certificate for domains "a_domain.com" : unable to generate a certificate for the domains [a_domain.com]: acme: Error -> One or more domains had a problem. [a_domain.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://a_domain.com/.well-known/acme-challenge/ra01JKbw3Wv194BDVhjSeK_nkbFA-UVYqnhv08LUoM [2606:4700:3037::681b:a340]

Port 80 must be open for HTTP traffic over IPv4. I had mine restricted to IPv6 (by mistake) and allowing IPv4 traffic on 80 resolved it.

[gantheaume]

I have literally done the install dozens of times and it never worked. Which instructions did you follow?

Sorry for my late answer, I’m quite busy at the moment; Here is precisely all I did, from a clean Ubuntu server 18.04 install:

If your user hasn’t the sudo rights:

su
usermod -a -G sudo yourusername
exit

From now on, everything is run from the normal user “yourusername”:

sudo apt-get update
sudo apt-get upgrade  ## Enter on all dialogs if there are some
sudo dpkg-reconfigure locales ## to have locals set up properly and stop having LC errors; I chose EN-US utf8
sudo apt-get install linux-headers-generic ethtool libc-dev linux-libc-dev python3-dev
sudo reboot

Now all is ready, we can do:

sudo ls ## just to have the sudo password entered
curl https://raw.githubusercontent.com/jupyterhub/the-littlest-jupyterhub/master/bootstrap/bootstrap.py | sudo -E python3 - --admin myfirstadminuser ## that's precisely the command of the install instructions in the manual: http://tljh.jupyter.org/en/latest/install/custom-server.html

Then, get things going; I don’t know if it’s all needed:

export PATH=/opt/tljh/user/bin:${PATH}
nano ~/.bashrc && source ~/.bashrc  ## Added the export path from above; source: http://tljh.jupyter.org/en/latest/howto/env/user-environment.html
sudo env PATH=${PATH} conda update -n base conda ## do not forget the "env"; it's actually missing from the tutorial page above, I'll think about editing it.

At last, the normal SSL procedure from this page: http://tljh.jupyter.org/en/latest/howto/admin/https.html

sudo tljh-config set https.enabled true
sudo tljh-config set https.letsencrypt.email [e-mail] ## more precisely, my email is hosted on mydomain.me, but I don't think it's important
sudo tljh-config add-item https.letsencrypt.domains mydomain.me
sudo tljh-config add-item https.letsencrypt.domains www.mydomain.me
sudo tljh-config show

When all is good: sudo tljh-config reload proxy

Now if you configured the DNS records properly (see my previous long post), all should go fine, and going to “mydomain.me” should bring you directly on the login secured with https 😉

Good luck testing 😉

Note that i already had a working https setup on the same domain using the universal letsencrypt procedure {my long post above) but I then wiped everything at started with a new ubuntu install, so it should not affect anything. Second, all this was part of quite a bit of trial and error, so you’re welcome to suggest improvements!

(By the way, it seems that the only reliable way of installing extra python modules is to use the command sudo -E pip intall module in the jupyter notebook terminal online! ‒and doing a sudo -E pip install --upgrade pip before‒. I didn’t manage any install of working modules any other way ‒for example through ssh‒. When I have time I’ll dig this, as it’s another issue. Linked help page, that details the steps: http://tljh.jupyter.org/en/latest/howto/env/user-environment.html)