Raise error "IndexError: index out of range in self" using NeighborLoader

🐛 Describe the bug

I used the neighbor_loader to sample my graph data. Although i used proper data, raise this error.

Usage is this:

NeighborLoader(
            processed_pyg_data,
            batch_size=self.args.batch_size,
            num_neighbors=[
                16,
                32,
            ]
        )

I think that my dataset is clean and unproblematic. Because i used this dataset in other projects with neighbor_loader at M1 Sillicon environment. But, in the ubuntu, raise this error below. What’s wrong… Plz help me…

  File "/home/ubuntu/anaconda3/envs/kisa-test/lib/python3.8/site-packages/torch_geometric/loader/base.py", line 36, in __next__
    return self.transform_fn(next(self.iterator))
  File "/home/ubuntu/anaconda3/envs/kisa-test/lib/python3.8/site-packages/torch_geometric/loader/neighbor_loader.py", line 405, in filter_fn
    data = filter_data(self.data, node, row, col, edge,
  File "/home/ubuntu/anaconda3/envs/kisa-test/lib/python3.8/site-packages/torch_geometric/loader/utils.py", line 158, in filter_data
    filter_node_store_(data._store, out._store, node)
  File "/home/ubuntu/anaconda3/envs/kisa-test/lib/python3.8/site-packages/torch_geometric/loader/utils.py", line 111, in filter_node_store_
    out_store[key] = index_select(value, index, dim=dim)
  File "/home/ubuntu/anaconda3/envs/kisa-test/lib/python3.8/site-packages/torch_geometric/loader/utils.py", line 26, in index_select
    return torch.index_select(value, dim, index, out=out)
IndexError: index out of range in self

Environment

• PyG version: 2.0.4
• PyTorch version: 1.12.0
• OS: Ubuntu
• Python version: 3.8
• CUDA/cuDNN version: None (CPU)
• How you installed PyTorch and PyG (conda, pip, source): pip
• Any other relevant information (e.g., version of torch-scatter):

torch-cluster 1.6.0 torch-geometric 2.1.0.post1 torch-scatter 2.0.9 torch-sparse 0.6.15 torch-spline-conv 1.2.1

[Padarn]

I’m wondering if we should add some better error messages around these kinds of common problems (I’ve seen a question similar to this quite a few times).

[davidlambda256]

Ok. I’ll give an example. The “damage” happens in this case below:

[STEP1] Preprocessing and Save (environment: Mac M1 sillicon)

        node_numerical_features = self._get_numerical_features(kind=kind)
        edge_structure, weight = self._get_graph_structure_features(kind=kind)
        labels = self._get_encoded_labels(kind=kind)

        processed_dataset = Data(
            x=node_numerical_features,
            edge_index=edge_structure,
            edge_attr=weight,
            y=labels,
        )

        torch.save(processed_dataset, "pyg_data.pt")

Then, i save the processed data using pyg, and load it in step2

[STEP2] Load the saved data (environment: Ubuntu 18.04)

       torch.load("pyg_data.pt")

The damage happens in STEP2. When i identified the loaded data, edge_index is damaged (e.g. Some elements in the loaded data changed negative values) Currently, i cannot find causes of this error. If i identify this issue, i’ll share you.