Phrasematcher; OverflowError: Python int too large to convert to C long

How to reproduce the behaviour

Greetings;

I am trying to use Spacy PhraseMatcher with token attr 'IS_SENT_START'. I guess like other attributes like ‘IS_PUNCT’ it should be possible to use the is_sent_start attr yet I got the following error.

`matcher = PhraseMatcher(nlp.vocab, attr = "IS_SENT_START")
Traceback (most recent call last):

  File "<ipython-input-74-c8e7d50d6aec>", line 1, in <module>
    matcher = PhraseMatcher(nlp.vocab, attr = "IS_SENT_START")

  File "phrasematcher.pyx", line 63, in spacy.matcher.phrasematcher.PhraseMatcher.__init__

OverflowError: Python int too large to convert to C long`

Here is the sample code;

import spacy
nlp = spacy.load('en_core_web_sm')
from spacy.matcher import PhraseMatcher
sub_list = ['however', 'although', 'on the other hand', 'nonetheless']
patterns = [nlp.make_doc(text) for text in sub_list]
matcher = PhraseMatcher(nlp.vocab, attr = 'IS_SENT_START')

Why so? I am trying to see if phrases in list are sent_start = True or False. No problem with Matcher yet it is not working with Phrasematcher. Can you help me with the solution or a work around?

Your Environment

• Operating System: macOS Catalina
• Python Version Used: 3.7.9
• spaCy Version Used: 2.0
• Environment Information: Anaconda-Spyder

[svlandeg]

Hi!

The error the matcher throws should definitely be more user-friendly, and the documentation should be updated too.

But you can make this work with attr='SENT_START'. Note that this will make the PhraseMatcher look at sentence segmentation, and so you’ll have to compile your patterns with nlp() instead of nlp.make_doc(text) to ensure that the parser runs (which does the sentence segmentation in en_core_web_sm)

This should work:

import spacy
from spacy.matcher import PhraseMatcher

nlp = spacy.load('en_core_web_sm')
sub_list = ['This is nice. Extraordinary.']
patterns = [nlp(text) for text in sub_list]    

The pattern will now be [True, None, None, None, True, None]

matcher = PhraseMatcher(nlp.vocab, attr='SENT_START')
matcher.add("TerminologyList", None, *patterns)
text = "It is awesome. Great. Just great. What I think? Awesome."
doc = nlp(text)
matches = matcher(doc)
for match_id, start, end in matches:
    span = doc[start:end]
    print(start, end, "-->", span.text)

This prints

0 6 --> It is awesome. Great. 
9 15 --> What I think? Awesome.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.