Spacy behaves differently when testing one case vs testing all cases

It seems Spacy’s tokenizer behaves differently when I run pytest -s --t=emojify and pytest -s --t=light --f=light.

For example, I added the following snippet in my generate() function:

print([str(t) for t in self.nlp(sentence)])

With input sentence "Apple is looking at buying U.K. startup for $132 billion."

pytest -s --t=emojify gives:

['Apple', 'is', 'looking', 'at', 'buying', 'U.K.', 'startup', 'for', '$', '132', 'billion', '.']

However, pytest -s --t=light --f=light gives:

['Apple', 'is', 'looking', 'at', 'buying', 'U.K.', 'startup', 'for', '$1', '32', 'billion.']

I use the fowling code to load spacy:

import spacy
from initialize import spacy_nlp
self.nlp = spacy_nlp if spacy_nlp else spacy.load("en_core_web_sm")

It looks very strange. Am I overlooking something?

[AbinayaM02]

Thank you for the answer! I have updated my branch again (fetch & merge), and the tests work locally. I can’t run the workflow in my PR, though, as this is my first contribution.

I have started the workflow. Let’s see if it goes through now.

[AbinayaM02]

Hello @AbinayaM02 I see that the first test fails, it wasn’t changed:

E           AssertionError: Mis-match in expected and predicted output for SuspectingParaphraser transformation: 
E              Expected Output: Sally finally returned the french book to Chris, didn't she? 
E              Predicted Output: Sally finally returned the french book to Chris, didn't it?
E           assert "Sally finall...s, didn't it?" == "Sally finall..., didn't she?"
E             Skipping 46 identical leading characters in diff, use -v to show
E             - s, didn't she?
E             ?           ^^^
E             + s, didn't it?
E             ?           ^^

I pulled your branch and tried running the test for suspecting_paraphraser only, it passes but the test run for all light testcases fails for suspecting_paraphraser. But when I run both the tests in my cloned repository, it is successful! Could you try “fetch & merge” your main branch [Fetch upstream on Github UI] and then pull the main branch into your shuffle_within_segements branch and try the pytest again?