Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

bcast/allreduce refuse to process with rank-1 size-1 tensor with non one stride.

See original GitHub issue

In pytorch, torch.tensor([1.+2.j]).real.contiguous().stride() returns (2,) instead of (1,), but the data in memory is contiguous indeed because of its size is only one. while bcast/allreduce of mpi4py raise an error saying it is not contiguous because of stride mismatch.

also see: https://github.com/pytorch/pytorch/issues/82610, they say it is contiguous already so nothing more should be done.

Issue Analytics

State:
Created a year ago
Comments:11 (8 by maintainers)

Top GitHub Comments

1reaction

dalcinlcommented, Aug 3, 2022

Fixed via 9ccf55fb0a0218d3fcc5eb7292ed14117d781852.

1reaction

hzhangxyzcommented, Aug 3, 2022

The real problem here is, in general, product(shape) = size = 1 arrays. mpi4py should accept all of these as contiguous. Am I missing something?

It seems the stride value of the same index should be ignored if any of element in shape = 1, not only product(shape) = 1

Top Results From Across the Web

RuntimeError: view size is not compatible with input tensor's ...

RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(....

What does .contiguous() do in PyTorch? - Stack Overflow

There are a few operations on Tensors in PyTorch that do not change the contents of a tensor, but change the way the...

Introduction to Tensors | TensorFlow Core

First, create some basic tensors. Here is a "scalar" or "rank-0" tensor . A scalar contains a single value, and no "axes". #...

Eigen Tensors - Eigen-unsupported

A TensorMap is not resizable because it does not own the memory where its data are stored. Constructor TensorMap<Tensor<data_type, rank>>(data, size0, size1, .....

torch.Tensor.view — PyTorch 1.13 documentation

For a tensor to be viewed, the new view size must be compatible with its original size and stride, i.e., each new view...