Windows support

Thanks for open sourcing this library. I was quite excited to take it for a spin, only to get an error “no module named ‘fcntl’”, and learning through #10, #23 and #46 that Windows is not supported, and there are no active plans for Windows support.

That is of course fine, but I have a few related questions.

• I see #10 has a wontfix and #46 has a help wanted label. That begs the question; would you be open to accepting contributions that add Windows support?

• Do you know what the major technical obstacles to Windows support are?

• Just now I see on the Installing Metaflow page “Metaflow is available as a Python package for MacOS and Linux.” Perhaps if it was followed by a more explicit “Windows is not supported.”, fewer people would miss this.

• Would it be suitable to place this on the roadmap, perhaps stating that there are no Netflix plans but outside contributions are welcome?

[savingoyal]

@visr, thanks for opening the issue -

• Windows support would be a complex undertaking (given our liberal use of multiprocessing for starters). We are definitely open to contributions and a good way to kick start the conversation would be to do a POC.
• We are updating our contributing guidelines to reflect this. Expect some progress on that front in the next couple of days.

[pombredanne]

@savingoyal re:

Windows support would be a complex undertaking (given our liberal use of multiprocessing for starters). We are definitely open to contributions and a good way to kick start the conversation would be to do a POC.

FWIW, all data scientists may not be running a *nix Linux or mac. Having official support for Windows would be a big plus. I wish I could coerce them though but that’s a dream. (And FWIW, WSL or Cygwin are their own can of frankenstein worms)

Multiprocessing may not be the main issue IMHO but rather:

1. keeping Python 2 and 3 compatibility is going to be a major pain on Windows

2. using signals such as here https://github.com/Netflix/metaflow/blob/c2f70c73782549fdce6c418410ecd9b305199b28/metaflow/plugins/timeout_decorator.py

3. possibly some assumptions about calling some functions and file system semantics that do not apply or are not available on Windows

For 2. Windows is a pain but I have some POSIX/windows utilities that have been battle tested for timeouts here https://github.com/nexB/scancode-toolkit/blob/develop/src/scancode/interrupt.py

For 1. we did that in https://github.com/nexB/scancode-toolkit and I would never do it again which would mean to support Python 3 for Windows. Anything else is a sure way to destroy grey matter very quickly 😉