Segmentation Fault with Pandas

I ran into a very odd segmentation fault error. This could very well be a PyTorch bug, but I thought I’d bring it up here, first. I’ve produced a minimal example at the bottom of this issue.

So far, I know that the fault happens at the loss.backward() call in model.fit(). The fault only seems to happen under the combination of two conditions (that I can find, so far):

1. When sparse=True.
2. Pandas is imported at the top of the file

(BTW, I pass in an SGD optimizer because that seems to be the only one that works right now with sparse embeddings)

I’m using pandas version 0.20.3 from conda, the latest spotlight from master, and PyTorch 0.2.0 from conda. I’d love to know if others can reproduce this.

As I said, this could very well be a PyTorch bug, but, if others run into this, it’ll be helpful to have this issue as a reference.

import pandas as pd
import numpy as np
import torch

from spotlight.interactions import Interactions
from spotlight.factorization.implicit import ImplicitFactorizationModel

user_ids = [2471, 5808, 3281, 4086, 6293, 8970, 11828, 3281]
item_ids = [1583, 57, 6963, 867, 8099, 10991, 24, 800]
num_users = 15274
num_items = 25655

train = Interactions(np.array(user_ids, dtype=np.int64),
                     np.array(item_ids, dtype=np.int64),
                     num_users=num_users,
                     num_items=num_items)

def optimizer_func(params, lr=0.01):
    return torch.optim.SGD(params, lr=lr)
  
RANDOM_STATE = np.random.RandomState(42)
model = ImplicitFactorizationModel(loss='bpr',
                                   embedding_dim=32,
                                   batch_size=4,
                                   n_iter=1,
                                   use_cuda=False,
                                   optimizer_func=optimizer_func,
                                   sparse=True,
                                   random_state=RANDOM_STATE)
# Fault
model.fit(train, verbose=True)

[apaszke]

A fix will be soon in master - https://github.com/pytorch/pytorch/pull/2581 Sorry for keeping you waiting!

[maciejkula]

@apaszke Awesome. Thanks for the amazing work on PyTorch!