Supporting manual configuration of SSL Client Certs

Hello all, this is somewhat related to #304

I am trying to build a custom orchestration/spawner/launcher system for deploying a distributed cluster. Ideally this system would manage session security, to prevent arbitrary clients from connecting to a scheduler. (We’re interested in this from the perspective of a multi-user host.)

Because this is all based on Tornado, it would be very convenient to be able to just use ssl_options and communicate SSL Client Certificates before launching anything in distributed. However this doesn’t work for the Client object, as this keyword isn’t exposed. It looks like ssl_options would need to be propagated to the Client somehow from this line up. (And maybe for Scheduler and Worker as well, I haven’t read the those implementations very closely yet)

Based on reading issues/blogs/etc it looks like managing security isn’t an immediate (or even necessarily future) goal of distributed/dask, but would it be acceptable to expose a little more of Tornado’s framework to permit programmatic management of SSL for more custom deployments? Ideally distributed could just pass some kwargs around and not consider the problem further.

If you’re amenable to the idea, I would be happy to work towards a PR (given a little API guidance).

Edit: I see that there is an existing PR #537 that sort of addresses this, but it looks like it may have stalled out a little, and it doesn’t seem to have a way to pass the ssl_options into TCPClient.connect in the same way that would be natural for the TCPServer objects.

[mrocklin]

Definitely amenable to a PR helping dask.distributed to pass through Tornado keywords 😃

[pitrou]

I can think of two options:

• either accept a SSLContext parameter (and only that)
• or accept a new kind of “security” object, that could define a SSL context, but possibly hold other parameters later (such as authentication, capabilities, etc.)

Either way, I am busy refactoring the I/O layer, so it’s not a good idea to start work on this immediately.