Overlapping multi-word phrases break text representation

When two bigrams from the gazetteer overlap in the text, the text representation of the first one is broken. Consider the following code:

import spacy.en
import spacy.matcher
from spacy.attrs import ORTH, TAG, LOWER, IS_ALPHA, FLAG63

patterns = [
    [{LOWER: 'food'}, {LOWER: 'safety'}],
    [{LOWER: 'safety'}, {LOWER: 'standards'}],
]

nlp = spacy.en.English(tagger=False, parser=False, load_vectors=False)

nlp.matcher.add('FOOD', 'FOOD', {}, patterns)

docs = nlp('There are different food safety standards in different countries.')

for e in docs.ents:
    print(e.text, e.label_)

Output:

>>> food FOOD
>>> safety standards FOOD

[honnibal]

Thanks! I’ll take a look at this next week.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.