Access dashboard from scheduler with no public IP?

Is it possible to connect to a scheduler without a public IP running in the same AWS VPC?

What happened:

I’m running a jupyter lab on an AWS EC2 instance and I’m trying to connect to a scheduler with a private IP that was created with dask-cloudprovider running in the same VPC as the lab.

The dashboard gets picked up by the extension as expected, all buttons become orange:

But if I open any of them, they remain blank for a while until they error out with a “took too long to respond” message:

What you expected to happen:

The extension showing the requested plots and metrics.

Anything else we need to know?:

Accessing the dashboard from the lab terminal with curl works, showing that it’s not a basic networking issue:

Nothing apparent in the console:

Extensions:

jovyan@201435007e31:~$ jupyter serverextension list
config dir: /usr/local/etc/jupyter
    dask_labextension  enabled 
    - Validating...
      dask_labextension 5.0.1 OK
    jupyter_server_proxy  enabled 
    - Validating...
      jupyter_server_proxy  OK
    jupyterlab  enabled 
    - Validating...
      jupyterlab 3.0.16 OK

Environment:

• Dask version: 2021.5.0
• Python version: 3.8.8
• Operating System: Ubuntu Bionic
• Install method (conda, pip, source): pip

[valpesendorfer]

thanks @ian-r-rose … required some fiddeling but it works now! Not going into details as this is more a proxy thing. I’d leave this issue open since this might be integrated into the extension. Feel free to close. Thanks!

[ian-r-rose]

Thanks for the additional detail @valpesendorfer.

I don’t expect hot-reloading to work. You will typically need to make sure that the configuration is present before the serve starts up (some hosted systems allow you to configure a start script or similar, though I’m not familiar with how sagemaker does it).