OverflowError: Python int too large to convert to C long

I am testing limits of dd.concat by passing a list of 1000 dask.dataframes each 10M rows:

Traceback(selected)

bcolz_stack_ = dd.concat(list(dask.compute(*concat_dd_list)))
output_dd = input_dd.set_index('global_id', drop=False)

dataframe\core.py in set_index()
return set_index(...)

dataframe\shuffle.py in set_index()
._repartition_quantiles(...)

dask\async.py in get_async()
raise(remote_exception(res,tb))
OverFLowError: Python int too large to convert to C long

dask\async.py, line 249, in _execute_task
return func(*args2)

It worked fine for 100 dask.dataframes. I am working on Windows 64-bit.

[ankravch]

Hmm, changing

a_dd = da.arange(start,stop, chunks=[chunksize])

to

a_dd = da.arange(start,stop, chunks=[chunksize], dtype=np.int64)

makes it work on Windows 64 bit.

[ankravch]

Or this one: https://github.com/paramiko/paramiko/issues/353 ‘The issue is with the low-level protocol decode – anything that’s a uint32 can be incorrectly decoded, or worse cause an integer overflow exception, if the MSB has the value -xff. The window size is sent as a uint32.’