`cupy.cumsum()` doesn't work in ROCm 5.0

Rel #6459.

cupy.cumsum() that CuPy independently implements without calling CUB does not work in ROCm 5.0:

$ python -c 'import cupy; print(cupy.cumsum(cupy.array([0,0])))'
[                  0 1304722626565153334]

The implementation uses shared memory to communicate across threads. As discussed in #4366, in ROCm, threads in a warp run in lock-step at all time so synchronizing instructions in a warp is not required, but we still need to use memory fence to enforce the ordering on access to shared memory.

[takagi]

No, I understand it doesn’t. In the following part of cumsum, for example, the data stored by thread 0 is observed being written by thread 1 is not necessarily before thread 1 observes the memory by the load instruction. https://github.com/cupy/cupy/blob/812b0f5301de8896f105ed974d84b03fcb331d91/cupy/_core/_routines_math.pyx#L307-L309

I couldn’t find its exact explanation in ROCm documentation, but CUDA tells that here: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#memory-fence-functions

[emcastillo]

Thanks! If hip supports memory fences maybe it is better to redefine it as such …, it will keep the semantics since the warp advances in lock-step