`test_delta_from_def_2` fails on aarch64, armv7 and ppc64le

Describe the issue: test_delta_from_def_2 fails on aarch64, armv7 and ppc64le.

[  371s] =================================== FAILURES ===================================
[  371s] ____________________________ test_delta_from_def_2 _____________________________
[  371s] [gw1] linux -- Python 3.8.15 /usr/bin/python3.8
[  371s] 
[  371s]     def test_delta_from_def_2():
[  371s]         # https://github.com/apache/parquet-format/blob/master/Encodings.md#example-2
[  371s]         expected = [7, 5, 3, 1, 2, 3, 4, 5]
[  371s]         data = np.zeros(1000, dtype="uint8")
[  371s]         o = cencoding.NumpyIO(data)
[  371s]         cencoding.encode_unsigned_varint(8, o)  # block size
[  371s]         cencoding.encode_unsigned_varint(1, o)  # miniblock count
[  371s]         cencoding.encode_unsigned_varint(8, o)  # value count
[  371s]         cencoding.encode_unsigned_varint(zigzag(7), o)  # first value (zigzag)
[  371s]     
[  371s]         # one and only miniblock
[  371s]         cencoding.encode_unsigned_varint(zigzag(-2), o)  # minimum delta (zigzag)
[  371s]         o.write_byte(2)  # bit-width list (only one)
[  371s]         o.write_byte(0b00000011)  # [0, 0, 0, 3]
[  371s]         o.write_byte(0b11111100)  # [3, 3, 3, pad]
[  371s]     
[  371s]         o.seek(0)
[  371s]     
[  371s]         outdata = np.zeros(1000, dtype="int32")
[  371s]         out = cencoding.NumpyIO(outdata.view("uint8"))
[  371s]         cencoding.delta_binary_unpack(o, out)
[  371s] >       assert outdata[:8].tolist() == expected
[  371s] E       assert [7, 5, 3, 1, -1, -3, -5, -7] == [7, 5, 3, 1, 2, 3, 4, 5]
[  371s] E         At index 4 diff: -1 != 2
[  371s] E         Full diff:
[  371s] E         - [7, 5, 3, 1, 2, 3, 4, 5]
[  371s] E         ?              ^     ^  ^
[  371s] E         + [7, 5, 3, 1, -1, -3, -5, -7]
[  371s] E         ?              ^^  +   ^^  ^^
[  371s] 
[  371s] ../../BUILDROOT/python-fastparquet-0.8.3-1.1.aarch64/usr/lib64/python3.8/site-packages/fastparquet/test/test_encoding.py:176: AssertionError

Minimal Complete Verifiable Example:

# Put your MCVE code here

Anything else we need to know?:

Environment:

• Dask version:
• Python version: 3.8
• Operating System: openSUSE Tumbleweed
• Install method (conda, pip, source): source

[martindurant]

Thanks for the pointer - I tracked it down. It seems that char means “signed char” on x86 and “unsigned char” on aarch; so I use the more specific int8_t.

[ggardet]

Do you happen to know which machine type on EC2 or GCE is best for running this kind of test? Or can I set up docker on a mac or linux to emulate it on M1/x86?

It can be reproduced on AWS A1 for sure and likely on M6g or Raspberry Pi 3/4.