dask-scheduler and dask-worker CLI break with `click==8.0.0`

What happened:

When using click=8.0.0 which was released on May 11th, the scheduler as well as worker CLI don’t come up.

What you expected to happen:

Working scheduler and worker CLIs

Minimal Complete Verifiable Example:

$ dask-scheduler --version
ValueError: 'default' must be a list when 'multiple' is true.
$ dask-worker --version
AttributeError: module 'click._unicodefun' has no attribute '_verify_python3_env'

Anything else we need to know?:

Probable root cause for dask-scheduler:

Click changelog:

The default value for a parameter with nargs > 1 and multiple=True must be a list of tuples. #1649

https://github.com/dask/distributed/blob/5e150aa6a3bd7decd38193a3d754497f8914d83c/distributed/cli/dask_scheduler.py#L104

dask-worker doesn’t set a default for multiple options and therefore is not affected by this, but shows a different failure.

Probable root cause for dask-worker:

https://github.com/dask/distributed/blob/5e150aa6a3bd7decd38193a3d754497f8914d83c/distributed/cli/utils.py#L22

Environment:

• pip install distributed reproduces the issue
• Click version needs to be: 8.0.0

[quasiben]

@ManuelBahr FYI, a patch was submitted to conda-forge to restrict click <8 for versions of dask older than 2021.05.0: https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/142/

[jrbourbeau]

Thanks for reporting @ManuelBahr and investigating @quasiben! I opened up a PR over in https://github.com/dask/distributed/pull/4810 which proposes we add some compatibility code to handle the changes in click 8