question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RuntimeError: Trying to create tensor with negative dimension -25: [-25]

See original GitHub issue

训练过程报错

  • 为了测试,我收集了一份简单的日志,并按照要求把文件转换成recbole指定的格式,但是运行的时候报错:RuntimeError: Trying to create tensor with negative dimension -25: [-25]
  • 另外时间戳的单位是(s or ms?)

以下是我的代码

  1. test.yaml
USER_ID_FIELD: user_id
ITEM_ID_FIELD: item_id
RATING_FIELD: rating
TIME_FIELD: timestamp
load_col:
    inter: [rating, user_id, item_id, timestamp]

min_user_inter_num: 10
min_item_inter_num: 10
lowest_val:
    rating: 2
eval_setting: RO_RS,full
split_ratio: [0.8,0.1,0.1]
  1. run.py
from recbole.config import Config
from recbole.data import create_dataset,data_preparation
from recbole.model.general_recommender import BPR
from recbole.trainer import Trainer
from logging import  getLogger
from recbole.utils import init_seed,init_logger

if __name__ == '__main__':
    # 初始化配置
    config = Config(model='BPR',dataset='test')
    config['data_path'] = './test_data/'
    config['dataset'] = 'test'
    # 初始随机种子,确保实验的可重复性
    init_seed(config['seed'],config['reproducibility'])
    ## 初始化日志
    init_logger(config)
    logger = getLogger()
    ## 将配置信息写入日志
    logger.info(config)
    ## 数据集创建和筛选
    dataset = create_dataset(config)
    logger.info(dataset)
    ## 数据集划分
    train_data,valid_data,test_data = data_preparation(config,dataset)

    ## 模型载入和初始化
    model = BPR(config,train_data).to(config['device'])
    logger.info(model)
    ## 训练器加载和初始化
    trainer = Trainer(config,model)

    # 模型训练
    best_valid_score, best_valid_result = trainer.fit(train_data, valid_data)

    ## 模型评估
    test_result = trainer.evaluate(test_data)
    print(test_result)

以下为完整的报错截图 以下是我的数据集样式 OS information:

  • OS: windows
  • RecBole Version 不太清楚
  • Python Version 3.6.12
  • PyTorch Version 1.7.0
  • cudatoolkit Version 11.1

新手上路,麻烦大大们了

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:11 (1 by maintainers)

github_iconTop GitHub Comments

6reactions
batmanflycommented, Nov 20, 2020

谢谢你提供的数据,前面的修复并不完全,应该将RecBole库中recbole/trainer/trainer.py从第313行至第322行的

extra_len_list = np.subtract(self.tot_item_num, tmp_len_list).tolist()
padding_nums = self.tot_item_num * len(tmp_len_list) - np.sum(tmp_len_list)
padding_tensor = torch.tensor([-np.inf], dtype=scores.dtype, device=self.device).repeat(padding_nums)
padding_scores = torch.split(padding_tensor, extra_len_list)

final_scores = list(itertools.chain.from_iterable(zip(pos_scores, neg_scores, padding_scores)))
final_scores = torch.cat(final_scores)

setattr(interaction, 'pos_len_list', pos_len_list)
setattr(interaction, 'user_len_list', len(tmp_len_list) * [self.tot_item_num])

修改为:

final_scores_width = max(self.tot_item_num, max(tmp_len_list))
extra_len_list = np.subtract(final_scores_width, tmp_len_list).tolist()
padding_nums = final_scores_width * len(tmp_len_list) - np.sum(tmp_len_list)
padding_tensor = torch.tensor([-np.inf], dtype=scores.dtype, device=self.device).repeat(padding_nums)
padding_scores = torch.split(padding_tensor, extra_len_list)

final_scores = list(itertools.chain.from_iterable(zip(pos_scores, neg_scores, padding_scores)))
final_scores = torch.cat(final_scores)

setattr(interaction, 'pos_len_list', pos_len_list)
setattr(interaction, 'user_len_list', len(tmp_len_list) * [final_scores_width])

修改代码后代码可正常运行并返回预期效果,感谢耐心解答,对初学者来讲很棒的项目。再次感谢

谢谢你指出来我们代码中的问题,我们将持续完善我们的代码,并且保持快速反馈、快速修正的开发态度,也欢迎你将此工具推荐给其他人试用。刚刚发布,确实难免存在问题。我们承诺将持续改善。谢谢!

–赵鑫

3reactions
chenyushuocommented, Nov 19, 2020

谢谢你提供的数据,前面的修复并不完全,应该将RecBole库中recbole/trainer/trainer.py从第313行至第322行的

extra_len_list = np.subtract(self.tot_item_num, tmp_len_list).tolist()
padding_nums = self.tot_item_num * len(tmp_len_list) - np.sum(tmp_len_list)
padding_tensor = torch.tensor([-np.inf], dtype=scores.dtype, device=self.device).repeat(padding_nums)
padding_scores = torch.split(padding_tensor, extra_len_list)

final_scores = list(itertools.chain.from_iterable(zip(pos_scores, neg_scores, padding_scores)))
final_scores = torch.cat(final_scores)

setattr(interaction, 'pos_len_list', pos_len_list)
setattr(interaction, 'user_len_list', len(tmp_len_list) * [self.tot_item_num])

修改为:

final_scores_width = max(self.tot_item_num, max(tmp_len_list))
extra_len_list = np.subtract(final_scores_width, tmp_len_list).tolist()
padding_nums = final_scores_width * len(tmp_len_list) - np.sum(tmp_len_list)
padding_tensor = torch.tensor([-np.inf], dtype=scores.dtype, device=self.device).repeat(padding_nums)
padding_scores = torch.split(padding_tensor, extra_len_list)

final_scores = list(itertools.chain.from_iterable(zip(pos_scores, neg_scores, padding_scores)))
final_scores = torch.cat(final_scores)

setattr(interaction, 'pos_len_list', pos_len_list)
setattr(interaction, 'user_len_list', len(tmp_len_list) * [final_scores_width])
Read more comments on GitHub >

github_iconTop Results From Across the Web

RuntimeError: Trying to create tensor with negative ...
I am using TransfoXLModel. The problem arises when running the code below (if I do not fill in vocab_size=256, it works fine):. the...
Read more >
Trying to create tensor with negative dimension -256
Hi. in torch 1.7.0 , I can't debug this error. class DQN(nn.Module): def __init__(self, num_states, num_actions): super(DQN, self).
Read more >
Pytorch Quantization RuntimeError: Trying to create tensor ...
It seems that the observers introduced in the graph are trying to create an histogram of negative dimension. Here is the error: x...
Read more >
Pytorch问题及解决:RuntimeError: Trying to create tensor ...
背景(个人记录)学知识图谱的表示学习,准备先跑一下RE-Net,在预训练时,作者的batchsize是1024,因为我用的是1050ti,设成10都跑不了, ...
Read more >
apr2000cbe/01-tensor-operations
An short introduction about PyTorch and about the chosen functions. torch.ones() - To create a new tensor of a given shape, that is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found