Automatically detect available cores and memory

Currently if the user does not provide cores and memory options we provide an informative error message asking them to provide cores= and memory= keywords.

By inspecting the local machine, we might change this slightly in two ways:

1. Provide suggested values like "please provide the following keyword arguments cores=24, memory="500 GB"
2. We could do this automatically, and change the error to a warning

If I recall correctly we guessed cores and memory previously, but we decided to not do this because sometimes the login nodes were different than the compute nodes. Should we reconsider this?

[lesteve]

I am not too convinced by auto-detection I have to admit. Maybe having a more helpful error message with a few examples could help? Something like this:

Please provide cores and memory. Here are a few examples:
cluster = FooCluster(cores=1, memory='10GB')
each job you are starting afterwards, e.g. by using cluster.scale, will use one core and 10GB of memory.

cluster = FooCluster(cores=6, processes=2, memory='10GB')
each job you are starting afterwards, e.g. by using cluster.scale, will use 6 cores with 3 threads per process and 10GB of memory.

If you are familiar with your job scheduler, you can look at the output of cluster.job_script() to see the full script which is going to be submitted.

Better wording more than welcome!

I noticed another midly annoying thing so I opened https://github.com/dask/dask-jobqueue/issues/329.

[guillaumeeb]

Ah, most of the machines I deal with are pretty homogeneous. It sounds like I have a biased sample though.

Cheyenne and Summit are special references 😃. It maybe funding principles between US and Europe which differ. I’ve Seen a lot of non heterogeneous platforms here.