question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Interpreting negative scores from lightfm.LightFM.predict()

See original GitHub issue

I saw some explanation from here regarding interpreting negative scores from the model.predict() method but I wanted to clarify a few points with the experts just for everyone to see as well.

I understand that the predicted scores don’t really mean anything but only used as a means to rank. Let’s say I have called lightfm.LightFM.predict() like this:

predictions = model.predict(
    user_ids=np.array([0, 0, 0, 0, 0]),
    item_ids=np.array([0, 1, 2, 5, 100]),
    item_features=feature_matrix
)

This means that I am predicting the score for user 0 on 5 items, namely 0, 1, 2, 5, 100. Let’s say that my result is:

array([ 2.79359961,  2.76859665, -6.60331917, -0.56102526,  1.27920794])

Does this mean that for user 0, we would predict item 3 (-6.60331917) to be the top recommendation , followed by item 4 (-0.56102526)… and item 1 (2.79359961) to be the lowest recommendation?

Thanks in advance!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:5

github_iconTop GitHub Comments

3reactions
EthanRosenthalcommented, Apr 24, 2020

Almost. Everything you said is right, but the conclusion at the end is actually the reverse – the top recommendation is the item with the highest predict score, not the lowest. In your case, item 1 would be the top recommendation, and item 3 would be the lowest.

2reactions
EthanRosenthalcommented, Apr 24, 2020

The scores are local to the individual user, so unfortunately you can’t compare scores between users. You’re correct in what you said – those scores are only used for ranking items for user 0.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Interpreting results from lightFM - Stack Overflow
They only make sense in the context of defining a ranking over items for a given user, with higher scores denoting a stronger...
Read more >
LightFM 1.16 documentation
When multiplied together, these representations produce scores for every item for ... LightFM.predict() methods, they are implicitly assumed to be identity ...
Read more >
lightfm-rec/Lobby - Gitter
The idea of considering tiers of interactions for negative sampling is just ... scores via batched matrix multiplys (instead of the model.predict() method)....
Read more >
Interpreting results of lightFM (factorization machines for ...
The prediction scores themselves are not interpretable: they are simply a means of creating a recommended item ordering for the user.
Read more >
What Should I Watch Next: A Movie Recommendation Study
B. Output Analysis . ... Table 3: Metric scores for models except lightFM . ... it will accurately predict what kind of movies...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found