Flags undefined in tokenize
See original GitHub issueHi,
Can’t reproduce it properly, but sometimes when using FileHighlighter plugin we get flags undefined in tokenize. Which leads to RegExp exception, it tries undefinedg.
Since I can’t reproduce when debugging i propose the following:
var flags = pattern.pattern.toString().match(/[imuy]*$/)[0];
pattern.pattern = RegExp(pattern.pattern.source, flags + "g");
Replaced with:
var flags = pattern.pattern.toString().match(/[imuy]*$/)[0];
pattern.pattern = RegExp(pattern.pattern.source, (flags || "") + "g");
Regards, Johan
Issue Analytics
- State:
- Created 7 years ago
- Comments:6 (2 by maintainers)
Top Results From Across the Web
TypeError: Cannot read properties of undefined (reading 'start')
tokenizer, I just copy from ner sample code and do a little bit modifications for relation task. from typing import List, Optional, Union, ......
Read more >pyright/tokenizer.ts at main - GitHub
* Converts a Python program text stream into a stream of tokens. */. import Char from 'typescript-char ...
Read more >python - UnparsedFlagAccessError: Trying to access flag
This code looks a bit old to be used with Tensorflow 2.x. See here, the solution would be to downgrade to tensorflow 1.x....
Read more >8.2.4 Tokenization — HTML5 - W3C
When a start or end tag token is created, its self-closing flag must be unset (its other state is that it be set),...
Read more >Tokenize - Oracle Help Center
The Tokenize step forms the first set of Tokens - termed Base Tokens. ... CONTROL, PUNCTUATION, SYMBOL, ALPHA_UPPERCASE, ALPHA_LOWERCASE and UNDEFINED.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
The user-agent would suggest, that it is a 64bit Linux system running Chromium 47.0.2526.80 which is over a year old now. Do all these users have the same user-agent? Could it be some kind of crawler that uses a fake user-agent?
The only way for
flags
to becomeundefined
is if you change either the prototype ofRegExp
orString
. At least as far as I can see. I don’t think we will add these changes if you cannot reproduce the bug, or show a plausible way howflags
can beundefined
.Maybe you could add some debug code to your site that provides more information:
Please reopen if you can provide more information regarding this issue.