[💡SUG] Do you support the ability to extract recommendations after training and evaluation?

Is your feature request related to a problem? Please describe. After I train and evaluation, I am struggling to figure out how to run the models on data after-the-fast. For example,

• suppose I have a series of user IDs for which I’d like to get the top-10 rankings for. or
• suppose I have a user ID and an item ID, and would like to know what is the item’s positional ranking for that user

Because the DataLoader’s class has several other elements built in (e.g. negative sampling config, used IDs information), I am unsure how to create a “fresh” DataLoader with IDs that I’d like to generate rankings for.

Describe the solution you’d like An exposed interface/function to evaluate:

• scores and/or rankings of items, given a user ID
• score and position of an item, given an item ID and a user ID

Describe alternatives you’ve considered I’ve tried to “override” the test_data DataLoader object in the quickstart with my own data, but ran into indexing issues and figured I’d ask first.

Additional context N/A – Thank you for open-sourcing this! Really cool!

[chenyushuo]

Thanks for your insightful commment. Because we are refactoring related functions, these features will be added in the future versions. If you want to implement these features in the current version temporally, please refer to this example:

import numpy as np
import torch
from recbole.model.general_recommender.bpr import BPR
from recbole.config import Config
from recbole.data import create_dataset, data_preparation
from recbole.data.interaction import Interaction


@torch.no_grad()
def get_scores(uid_series, model, test_data):
    """Calculate the scores of all items for each user in uid_series.
    
    Note:
        The score of [pad] and history items will be set into -inf.
    
    Args:
        uid_series (np.ndarray): User id series
        model (AbstractRecommender): Model to predict
        test_data (AbstractDataLoader): The test_data of model
    
    Returns:
        torch.Tensor: the scores of all items for each user in uid_series.
    """
    uid_field = test_data.dataset.uid_field
    iid_field = test_data.dataset.iid_field
    dataset = test_data.dataset
    
    # Get scores of all items
    input_interaction = Interaction({uid_field: torch.tensor(uid_series.repeat(dataset.item_num))})
    input_interaction.update(test_data.get_item_feature().repeat(len(uid_series)))
    score = model.predict(input_interaction).view(len(uid_series), dataset.item_num)

    score[:, 0] = -np.inf  # set scores of [pad] to -inf

    # Get history items
    test_inter = test_data.dataset.inter_feat
    history_item_ids = []
    for uid in uid_series:
        pos_item_id = test_inter[iid_field][test_inter[uid_field] == uid].values
        used_item_id = test_data.sampler.used_ids[uid]
        history_item_ids.append(list(used_item_id - set(pos_item_id)))

    # set scores of history items to -inf
    for i, hist_iid in enumerate(history_item_ids):
        score[i, hist_iid] = -np.inf

    return score

def get_topk(uid_series, model, test_data, k):
    """Calculate the top-k items' scores and ids for each user in uid_series.
    
    Args:
        uid_series (np.ndarray): User id series
        model (AbstractRecommender): Model to predict
        test_data (AbstractDataLoader): The test_data of model
        k (int): The top-k items.
    
    Returns:
        tuple:
            - topk_scores (torch.Tensor): The scores of topk items.
            - topk_index (torch.Tensor): The index of topk items, which is also the internal ids of items.
    """
    score = get_scores(uid_series, model, test_data)
    return torch.topk(score, k)


if __name__ == '__main__':
    # Load config, dataset and model (if you have load these things, you can skip these codes)
    config = Config(model='BPR', dataset='ml-100k')
    dataset = create_dataset(config)
    train_data, valid_data, test_data = data_preparation(config, dataset)
    model = BPR(config, train_data)
    checkpoint = torch.load('RecBole/saved/BPR-Nov-16-2020_10-38-53.pth')
    model.load_state_dict(checkpoint['state_dict'])
    model.eval()

    # uid_series = np.array([1, 2])  # internal user id series
    # or you can use dataset.token2id to transfer external user token to internal user id
    uid_series = dataset.token2id(dataset.uid_field, ['196', '186'])

    topk_score, topk_iid_list = get_topk(uid_series, model, test_data, 10)
    print(topk_score)  # scores of top 10 items
    print(topk_iid_list)  # internal id of top 10 items
    external_item_list = dataset.id2token(dataset.iid_field, topk_iid_list)
    print(external_item_list)  # external tokens of top 10 items
    print()

    score = get_scores(uid_series, model, test_data)
    print(score)  # score of all items
    print(score[0, dataset.token2id(dataset.iid_field, ['242', '302'])])  # score of item ['242', '302'] for user '196'.

Note: method dataset.token2id & dataset.id2token are added in #511 .

Output:

[mayaKaplansky]

Hi is there a way to re-open this thread? there are 2 unanswered questions here. many thanks