You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Traceback (most recent call last):
File "reproduce.py", line 4, in <module>
reward_fn([[1, 2, 3]])
File "trl/rewards/other_rewards.py", line 78, in __call__
scaling = 1 - len(set(ngrams)) / len(ngrams)
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
ZeroDivisionError: division by zero
Negative values have the same problem because no n-grams are constructed.
Expected behavior: validate ngram_size when constructing the reward function and raise an actionable ValueError unless it is greater than zero. This is consistent with the existing construction-time validation for max_penalty.
System Info
Platform: macOS-26.6.2-arm64-arm-64bit-Mach-O
Python version: 3.14.6
TRL version: 1.13.0.dev0+323efb2
PyTorch version: 2.13.0
accelerator(s): cpu
Transformers version: 5.16.1
Accelerate version: 1.14.0
Accelerate config: not found
Datasets version: 5.0.1
HF Hub version: 1.29.0
bitsandbytes version: not installed
DeepSpeed version: not installed
Liger-Kernel version: not installed
PEFT version: not installed
vLLM version: not installed
Checklist
I have checked that my issue isn't already filed (see open issues)
I have included my system information
Any code provided is minimal, complete, and reproducible (more on MREs)
Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
ngram_size is the sliding-window size for building n-grams (zip(ids[0:], ids[1:], ...))
ngram_size=0: range(0) is empty, so the list comprehension produces [], and zip() with no arguments yields nothing. ngrams ends up as an empty list, so line 78 computes len(set([])) / len([]) — that's 0 / 0, the ZeroDivisionError in the issue.
ngram_size=-1: same thing, because range(-1) is also empty. The guard on line 74 (len(ids) < self.ngram_size) doesn't save us.
So there's no sensible interpretation for non-positive values; I'd propose raising a ValueError at construction (mirroring the existing max_penalty check).
That's not my question. My question is what does it means, ie why a user would choose such a value? In what real-world training would you use 0 or negative size?
Hi, there is no real world use of ngram_size=0 or negative value. Nobody will choose it on purpose. ngram_size is the size of the sliding window over token ids. A window of 0 tokens (or -1 token) dont mean anything. It was probably a typo or someone just testing edge case.
Example with ngram_size=2:
[1, 2, 3, 4] gives windows (1,2) (2,3) (3,4). All are diffrent, so no repetition.
[5, 5, 5, 5, 5] gives (5,5) (5,5) (5,5) (5,5). 4 windows but only 1 unique, so very repetitive.
Reward formula is (1 - unique / total) * max_penalty. So:
ids= [1, 2, 3]
[ids[i:] foriinrange(0)] # range(0) is empty -> []list(zip(*[])) # zip() with no argument -> []len(set([])) /len([]) # 0 / 0 -> ZeroDivisionError
So it fails for every input, even [[1, 2, 3]] which is a valid input. The input is not the problem, the size is the problem. Negative value also give empty window, same error.
So the issue is not about supporting some use case. It is only that the error come late and is confusing (ZeroDivisionError). Better to raise a ValueError in the constructor when ngram_size < 1, same like the max_penalty check. Thanks!
Thanks for digging into it. Agreed that ngram_size=0 or a negative value has no meaning and nobody picks it on purpose.
That is the reason not to add the check rather than a reason to add it. TRL avoids defensive validation for cases that do not occur, and a typo here already fails immediately and points at the line: the first call raises ZeroDivisionError at scaling = 1 - len(set(ngrams)) / len(ngrams). A guard would only reword an error the user already gets.
max_penalty is validated for the opposite reason: a positive value never raises. It silently flips the sign, so repetition is rewarded instead of penalised and training quietly optimises toward degenerate text. That is the case worth guarding, and it is the only such guard in trl/rewards/.
Activity
qgallouedec commentedon Sep 16, 2026
what does ngram_size=0 or -1 mean?
pravinmountain commentedon Sep 18, 2026
ngram_sizeis the sliding-window size for building n-grams (zip(ids[0:], ids[1:], ...))ngram_size=0:range(0)is empty, so the list comprehension produces[], andzip()with no arguments yields nothing.ngramsends up as an empty list, so line 78 computeslen(set([])) / len([])— that's0 / 0, theZeroDivisionErrorin the issue.ngram_size=-1: same thing, becauserange(-1)is also empty. The guard on line 74(len(ids) < self.ngram_size)doesn't save us.So there's no sensible interpretation for non-positive values; I'd propose raising a
ValueErrorat construction (mirroring the existingmax_penaltycheck).qgallouedec commentedon Sep 18, 2026
That's not my question. My question is what does it means, ie why a user would choose such a value? In what real-world training would you use 0 or negative size?
pravinmountain commentedon Oct 1, 2026
Hi, there is no real world use of
ngram_size=0or negative value. Nobody will choose it on purpose.ngram_sizeis the size of the sliding window over token ids. A window of 0 tokens (or -1 token) dont mean anything. It was probably a typo or someone just testing edge case.Example with
ngram_size=2:[1, 2, 3, 4]gives windows(1,2) (2,3) (3,4). All are diffrent, so no repetition.[5, 5, 5, 5, 5]gives(5,5) (5,5) (5,5) (5,5). 4 windows but only 1 unique, so very repetitive.Reward formula is
(1 - unique / total) * max_penalty. So:[1, 2, 3, 4]->1 - 3/3 = 0-> reward0.0[5, 5, 5, 5, 5]->1 - 1/4 = 0.75-> reward-0.75Now with
ngram_size=0:So it fails for every input, even
[[1, 2, 3]]which is a valid input. The input is not the problem, the size is the problem. Negative value also give empty window, same error.So the issue is not about supporting some use case. It is only that the error come late and is confusing (
ZeroDivisionError). Better to raise aValueErrorin the constructor whenngram_size < 1, same like themax_penaltycheck. Thanks!albertvillanova commentedon Oct 1, 2026
Thanks for digging into it. Agreed that
ngram_size=0or a negative value has no meaning and nobody picks it on purpose.That is the reason not to add the check rather than a reason to add it. TRL avoids defensive validation for cases that do not occur, and a typo here already fails immediately and points at the line: the first call raises
ZeroDivisionErroratscaling = 1 - len(set(ngrams)) / len(ngrams). A guard would only reword an error the user already gets.max_penaltyis validated for the opposite reason: a positive value never raises. It silently flips the sign, so repetition is rewarded instead of penalised and training quietly optimises toward degenerate text. That is the case worth guarding, and it is the only such guard intrl/rewards/.Closing as not planned.