Option to prevent automatic rechunking?

In xhistogram #57 I’m trying to test a blockwise-based algorithm for various chunk shapes, and finding that in my test suite dask will change my tests by automatically rechunking and issuing a PerformanceWarning:

  /home/tegn500/Documents/Work/Code/xhistogram/xhistogram/core.py:334: 
  PerformanceWarning: Increasing number of chunks by factor of 100
    bin_counts = dsa.blockwise(

I would prefer for dask not to override me like this - in a test suite I’m much more concerned that the tests are run exactly the way I specify than I am concerned about performance.

Is there a global option to prevent this? My dask.config.config dictionary looks like this

{'version': 1,
 'temporary-directory': None,
 'dataframe': {'shuffle-compression': None},
 'array': {'svg': {'size': 120}, 'slicing': {'split-large-chunks': None}},
 'optimization': {'fuse': {'active': None,
   'ave-width': 1,
   'max-width': None,
   'max-height': inf,
   'max-depth-new-edges': None,
   'subgraphs': None,
   'rename-keys': True}}}

but I’m not sure if any of the options in the configuration reference will affect this.

It’s hard for me to know if my tests failing due to this or not. Some of my tests are failing, and when dask is automatically changing the test as it runs I don’t really know how to debug them. blockwise is dispatching to code we wrote so it’s plausible that the automatic rechunking is causing my test failures by switching to a chunking pattern which passes to a chunking pattern which fails.

The only issue I’ve seen that seems related is #4763 .

[quasiben]

@gjoseph92 your comment is fantastic! I’m not sure where this should go immediately but I think we should find a space in the docs to capture those clarifying thoughts long term

[TomNicholas]

Thank you for that clarification @gjoseph92 , that’s extremely helpful.

It does make me wonder how my test input has even got unaligned chunks, but that’s something to be discussed in https://github.com/xgcm/xhistogram/pull/57 rather than here I guess.