Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Incorrect warning about WhitespaceTokenizer
See original GitHub issueThe warning in https://rasa.com/docs/rasa/user-guide/evaluating-models/#intent-classification is not true anymore. Our WhitespaceTokenizer splits Brian's into multiple tokens.
We should update or remove the warning.
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (5 by maintainers)
Top Results From Across the Web
Incorrect warning about WhitespaceTokenizer #4605 - GitHub
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity...
Read more >Case insensitive whitespace tokenizer - Rasa Open Source
Hi, I found several forum posts advising people to use the following for implementing a case insensitive pipeline.
Read more >Tokenizer - Hugging Face
Tokenizer. A tokenizer is in charge of preparing the inputs for a model. The library contains tokenizers for all the models. Most of...
Read more >How to set whitespace tokenizer on NER Model?
You set your own tokenizer by specifying the classname to the tokenizerFactory flag/property: tokenizerFactory = edu.stanford.nlp.process.
Read more >AnalyzersTokenizersTokenFilters - Solr
Documentation at White Space Tokenizer. ... To avoid collisions and false matches, reversed tokens are indexed with a prefix that should not ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hi Can I have a go at this issue?
fixed in https://github.com/RasaHQ/rasa/pull/5643