Can we add an augment `min_new_tokens` to the `generate` function?
See original GitHub issueFeature request
Can we add a new parameter min_new_tokens
to the generate
function to limit the length of newly generated tokens? The current parameter min_length
limits the length of prompt + newly generated tokens
, not the length of newly generated tokens
.
Motivation
We already have max_new_tokens
to limit the max length of the generated tokens, i.e., max_length = max_new_tokens + prompt
.
Why not add the min_new_token
to limit the min length of the generated tokens? (i.e., min_length = min_new_tokens + prompt
)
I know this is kind like an other syntax sugar, but it will be much convenient if we have this parameter.
Your contribution
I can sumbit a PR.
Issue Analytics
- State:
- Created 9 months ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
How to activate an Augment in Fortnite and Augments list
You need to activate an Augment in Fortnite Chapter 4 Season 1 in order to gain some helpful buffs, as well as unlock...
Read more >Augmenting Function.prototype and make a method ...
I learned from Douglas Crockford's book that if you augment Function.prototype, you can make a method available to all functions.
Read more >Fortnite: All Augments And How They Work
First Assault - The first bullet in the magazine of your Assault Rifle deals bonus damage. Forecast - You can always see the...
Read more >Augmenting Types - JavaScript: The Good Parts [Book]
In Chapter 3, we saw that adding a method to Object.prototype makes that method ... For example, by augmenting Function.prototype , we can...
Read more >Augmentation Libraries - dart-lang/language
Augmentations can add new top-level declarations, inject new members into classes, and wrap functions and variables in additional code.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@PanQiWei I agree, but I think it is impossible to change the current implementation of
max_length
andmin_length
for the conern of back compatibility.@silverriver yes you are right, its my mistake 😂
But I still think
min_new_tokens
andmin_length
should mean the same thing and also tomax_new_tokens
andmax_length
(though they are actually different now), because most people who usemodel.generate
would thinkmin_length
means to ‘at least generate min_length tokens’ andmax_length
means to ‘generate tokens no more than max_length’