BinderHub setup on Azure timesout when trying to contact docker registry

When following zero to binderhub instructions using Azure, the following command:-

helm install jupyterhub/binderhub --version=0.2.0-3b53fce     --name=sheff-hub     --namespace=sheff-hub -f secret.yaml -f config.yaml

Produced this error message:-

Error: release sheff-hub failed: clusterroles.rbac.authorization.k8s.io "sheff-hub-image-cleaner" is forbidden: attempt to grant extra privileges: [{[get] [] [nodes] [] []} {[patch] [] [nodes] [] []}] user=&{system:serviceaccount:kube-system:tiller ...... [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]} ownerrules=[] ruleResolutionErrors=[[clusterroles.rbac.authorization.k8s.io "system:discovery" not found, clusterroles.rbac.authorization.k8s.io "cluster-admin" not found]]

We managed to fix this using this issue. However, when trying to use my own binderhub with an example repo I get an “Internal server error”. I have checked by dockerhub username and password and they are correct.

I can look at the log files with the command

kubectl logs binder-7dd4d6dc95-dzrvr -n sheff-hub

and the following error is seen

[I 190318 15:43:34 repoproviders:441] Using cached ref for https://api.github.com/repos/binder-examples/requirements/commits/master: a73ba121c9847fa38b7c4153230b9bfa9eecfaa7
[I 190318 15:43:38 web:2106] 200 GET /about (10.244.1.1) 1.24ms
[I 190318 15:43:43 web:2106] 200 GET /about (10.244.1.1) 1.14ms
[I 190318 15:43:48 web:2106] 200 GET /about (10.244.1.1) 1.22ms
[I 190318 15:43:53 web:2106] 200 GET /about (10.244.1.1) 1.32ms
[E 190318 15:43:54 web:1621] Uncaught exception GET /build/gh/binder-examples/requirements/master (10.240.0.6)
    HTTPServerRequest(protocol='http', host='40.74.53.217', method='GET', uri='/build/gh/binder-examples/requirements/master', version='HTTP/1.1', remote_ip='10.240.0.6')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/site-packages/tornado/web.py", line 1543, in _execute
        result = yield result
      File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1099, in run
        value = future.result()
      File "/usr/local/lib/python3.6/site-packages/binderhub/builder.py", line 273, in get
        image_manifest = await self.registry.get_image_manifest(*'/'.join(image_name.split('/')[-2:]).split(':', 1))
      File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1107, in run
        yielded = self.gen.throw(*exc_info)
      File "/usr/local/lib/python3.6/site-packages/binderhub/registry.py", line 199, in get_image_manifest
        auth_resp = yield client.fetch(auth_req)
      File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1099, in run
        value = future.result()
    tornado.curl_httpclient.CurlError: HTTP 599: Operation timed out after 20001 milliseconds with 0 bytes received

[alexmorley]

@betatim I don’t think we were using the values.yaml in the setup. But we did check the login details for dockerhub to make sure they were correct (and so I don’t think it’s #811).

[h4818]

check or remove token_url config item: config.DockerRegistry.token_url