Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
How can I access prompt scores/logprobs?
See original GitHub issueI have tried model.generate(**inputs, return_dict_in_generate=True, output_scores=True) but it only gives the scores for generated tokens. For my application, it would be convenient if there’s a similar parameter to echo in the OpenAI GPT-3 API that lets us access prompt scores/logprobs, but I have yet to find it. Any help is appreciated!
Issue Analytics
- State:
- Created a year ago
- Comments:8 (3 by maintainers)
Top Results From Across the Web
How can I access prompt scores/logprobs? - Transformers
I have tried model.generate(**inputs, return_dict_in_generate=True, output_scores=True) but it only gives the scores for generated tokens.
Read more >Logprobs - GPT_Prompts - Organizing GPT-3 Prompts - Wikidot
This section breaks down how to access the log probs and how to use them. ... Completion.create(prompt=prompt, **kwargs) scores = pd.
Read more >Getting the Most Out of GPT-3-based Text Classifiers: Part Three
In this example, the text in bold is the “prompt” and the rest of the text is ... You have to apply for...
Read more >GPT-3 davinci gives different results with the same prompt
I am not sure if you have access to GPT-3, particularly DaVinci (the complete-a-sentence tool). You can find the API and info here....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hey @xiaoyangnickhu 👋 We do have a tool for that, but which is not yet documented: 👉 compute_transition_beam_scores
Their arguments should be self explanatory, but let me know if you’d like further guidance 😃
@xiaoyangnickhu oops, you are absolutely right, you can obtain the conditional logits that way! (I’ve been so used to work on generate, one token at a time, that forgot that the
.logitsoutput holds the desired output for all steps).I’ve edited my answer above in case someone stumbles across this thread 😃