LocalCluster not respecting dask distributed config

Running dask.config.set to update the worker memory targets does not seem to affect the worker limits in Cluster, and the config is not respected (despite being represented in the print)

import dask
from dask.datasets import timeseries
from distributed import LocalCluster, Client
dask.config.set({'distributed.worker.memory.target': False, 'distributed.worker.memory.spill': False})
print(dask.config.config['distributed']['worker']['memory'])
cluster = LocalCluster(n_workers=4, threads_per_worker=2, memory_limit='1G')
client = Client(cluster)
dfs = []
for i in range(60):
    dfs.append(client.persist(timeseries()))

I would have expected the code above to cause the workers to die and have some errors about memory, but I’m getting nothing. And subsequent calls, eg.

for x in dfs:
    print(x.size.compute())

Now take lots of time (data didn’t fit in to memory, and is being loaded from disk).

How can I, in python, spin up a LocalCluster with no-spill settings?

[bluecoconut]

Awesome! thank you~

[mrocklin]

@zyxue I recommend raising a new issue with your situation