Windows support

When trying to run jupyterlab-nvdashboard on windows I’m getting the following error on startup.

(Sorry about the screenshot, copying from powershell seems to lose all linebreaks)

I imagine this is related to the way the server proxy is using asyncio/tornado to create subprocesses.

Conda env

name: pipeline
channels:
  - conda-forge
  - defaults
dependencies:
  - attrs=19.1.0=py_0
  - backcall=0.1.0=py_0
  - bleach=3.1.0=py_0
  - bokeh=1.3.4=py37_0
  - ca-certificates=2019.5.15=1
  - certifi=2019.6.16=py37_1
  - click=7.0=py_0
  - cloudpickle=1.2.2=py_0
  - colorama=0.4.1=py_0
  - cudatoolkit=10.0.130=0
  - cudnn=7.6.0=cuda10.0_0
  - cupy=6.0.0=py37h230ac6f_0
  - cytoolz=0.10.0=py37hfa6e2cd_0
  - dask=2.4.0=py_0
  - dask-core=2.4.0=py_0
  - decorator=4.4.0=py_0
  - defusedxml=0.5.0=py_1
  - distributed=2.4.0=py_0
  - entrypoints=0.3=py37_1000
  - fastrlock=0.4=py37h6538335_1000
  - freetype=2.10.0=h563cfd7_1
  - fsspec=0.5.1=py_0
  - heapdict=1.0.0=py37_1000
  - intel-openmp=2019.5=281
  - ipykernel=5.1.2=py37h5ca1d4c_0
  - ipython=7.8.0=py37h5ca1d4c_0
  - ipython_genutils=0.2.0=py_1
  - jedi=0.15.1=py37_0
  - jinja2=2.10.1=py_0
  - jpeg=9c=hfa6e2cd_1001
  - json5=0.8.5=py_0
  - jsonschema=3.0.2=py37_0
  - jupyter_client=5.3.1=py_0
  - jupyter_core=4.4.0=py_0
  - jupyterlab=1.1.4=py_0
  - jupyterlab_server=1.0.6=py_0
  - libblas=3.8.0=12_mkl
  - libcblas=3.8.0=12_mkl
  - liblapack=3.8.0=12_mkl
  - libpng=1.6.37=h7602738_0
  - libsodium=1.0.17=h2fa13f4_0
  - libtiff=4.0.10=h6512ee2_1003
  - llvmlite=0.29.0=py37hed17590_1
  - locket=0.2.0=py_2
  - lz4-c=1.8.3=he025d50_1001
  - m2w64-gcc-libgfortran=5.3.0=6
  - m2w64-gcc-libs=5.3.0=7
  - m2w64-gcc-libs-core=5.3.0=7
  - m2w64-gmp=6.1.0=2
  - m2w64-libwinpthread-git=5.0.0.4634.697f757=2
  - markupsafe=1.1.1=py37hfa6e2cd_0
  - mistune=0.8.4=py37hfa6e2cd_1000
  - mkl=2019.4=245
  - msgpack-python=0.6.2=py37he980bc4_0
  - msys2-conda-epoch=20160418=1
  - nbconvert=5.6.0=py37_1
  - nbformat=4.4.0=py_1
  - nodejs=10.13.0=0
  - notebook=6.0.1=py37_0
  - numba=0.45.1=py37hf9181ef_0
  - numpy=1.15.4=py37h8078771_1002
  - olefile=0.46=py_0
  - openssl=1.1.1d=he774522_0
  - packaging=19.2=py_0
  - pandas=0.25.1=py37he350917_0
  - pandoc=2.7.3=0
  - pandocfilters=1.4.2=py_1
  - parso=0.5.1=py_0
  - partd=1.0.0=py_0
  - pickleshare=0.7.5=py37_1000
  - pillow=6.1.0=py37h643dfcc_1
  - pip=19.2.3=py37_0
  - prometheus_client=0.7.1=py_0
  - prompt_toolkit=2.0.9=py_0
  - psutil=5.6.3=py37hfa6e2cd_0
  - pygments=2.4.2=py_0
  - pyparsing=2.4.2=py_0
  - pyrsistent=0.15.4=py37hfa6e2cd_0
  - python=3.7.3=h510b542_1
  - python-dateutil=2.8.0=py_0
  - pytz=2019.2=py_0
  - pywinpty=0.5.5=py37_1000
  - pyyaml=5.1.2=py37hfa6e2cd_0
  - pyzmq=18.1.0=py37h16f9016_0
  - send2trash=1.5.0=py_0
  - setuptools=41.2.0=py37_0
  - six=1.12.0=py37_1000
  - sortedcontainers=2.1.0=py_0
  - sqlite=3.29.0=hfa6e2cd_1
  - tbb=2019.8=he980bc4_0
  - tblib=1.4.0=py_0
  - terminado=0.8.2=py37_0
  - testpath=0.4.2=py_1001
  - tk=8.6.9=hfa6e2cd_1003
  - toolz=0.10.0=py_0
  - tornado=6.0.3=py37hfa6e2cd_0
  - traitlets=4.3.2=py37_1000
  - vc=14.1=h0510ff6_4
  - vs2015_runtime=14.16.27012=hf0eaf9b_0
  - wcwidth=0.1.7=py_1
  - webencodings=0.5.1=py_1
  - wheel=0.33.6=py37_0
  - wincertstore=0.2=py37_1002
  - winpty=0.4.3=4
  - xz=5.2.4=h2fa13f4_1001
  - yaml=0.1.7=hfa6e2cd_1001
  - zeromq=4.3.2=h6538335_2
  - zict=1.0.0=py_0
  - zlib=1.2.11=h2fa13f4_1006
  - zstd=1.4.0=hd8a0e53_0
  - pip:
    - aiohttp==3.6.1
    - async-timeout==3.0.1
    - chardet==3.0.4
    - idna==2.8
    - jupyter-server-proxy==1.1.0
    - jupyterlab-nvdashboard==0.1.9
    - multidict==4.5.2
    - pynvml==8.0.3
    - simpervisor==0.3
    - yarl==1.3.0
prefix: C:\Users\jacob\Miniconda3\envs\pipeline

[oeway]

I did a bit more research on the windows support, my conclusion is that we may need to implement a fallback solution for windows which does not use simpervisor.

The reason is:

The default asyncio event loop implementation on Windows does not support subprocesses. Subprocesses are available for Windows if a ProactorEventLoop is used. See Subprocess Support on Windows for details.

https://docs.python.org/3/library/asyncio-subprocess.html

I tried to switch the eventloop for simpervisor:

if sys.platform == 'win32':
    loop = asyncio.ProactorEventLoop()
    asyncio.set_event_loop(loop)

However, the entire jupyter server hangs, and I figured it maybe because tornado server doesn’t not support ProactorEventLoop:

https://github.com/tornadoweb/tornado/blob/d6819307ee050bbd8ec5deb623e9150ce2220ef9/tornado/platform/asyncio.py#L15-L19

We may need to implement a synchronous version of the SupervisedProcess, and run it in a thread/executor instead.

[ryanlovett]

Thanks for the report @jacobtomlinson. Yes, it seems that some asyncio features used by simpervisor are not supported on Windows. I don’t know if this can be addressed within simpervisor – that’s out of my depth. I think the most we can do here is document this issue.