Possible bug with punctuation in a text

Hi guys when running the mitie model with the following sample

{
  "rasa_nlu_data": {
    "common_examples": [
     
      {
        "text": "Hi, thank you for your interest, the last price I can go for it is 200,000.",
        "intent": "new_offer",
        "entities": [
          {
            "start": 0,
            "end": 2,
            "value": "Hi",
            "entity": "greeting"
          },
          {
            "start": 67,
            "end": 74,
            "value": "200,000",
            "entity": "price"
          }
        ]
      }
    ]
  }
}

i am getting this error

Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/site-packages/rasa_nlu/train.py", line 65, in <module>
    do_train(config)
  File "/usr/local/lib/python2.7/site-packages/rasa_nlu/train.py", line 59, in do_train
    trainer.train(training_data)
  File "/usr/local/lib/python2.7/site-packages/rasa_nlu/trainers/mitie_trainer.py", line 25, in train
    self.entity_extractor = self.train_entity_extractor(data.entity_examples)
  File "/usr/local/lib/python2.7/site-packages/rasa_nlu/trainers/mitie_trainer.py", line 42, in train_entity_extractor
    start, end = self.start_and_end(tokens, val_tokens)
  File "/usr/local/lib/python2.7/site-packages/rasa_nlu/trainers/mitie_trainer.py", line 31, in start_and_end
    start, end = locs[0], locs[0] + len(entity_tokens)
IndexError: list index out of range

I am not sure how to fix this, but i am positive it is related to the punctuation in in the text string, because when i remove the last ‘.’ in the training sample text field it works. Thanks for your amazing product!

[tmbo]

No they are not ignored, what happens to them is not defined. It depends on the entity extractor (mitie / spacy).

[mikkelam]

I am now just getting a warning,

[u'Hi', u',', u'thank', u'you', u'for', u'your', u'interest', u',', u'the', u'last', u'price', u'I', u'can', u'go', u'for', u'it', u'is', u'200,000.'].Entities must span whole tokens.

but what actually happens with this sample? Is it ignored?