Latin Lemmatizer raises an error on unknown form ?

Code :

from cltk.stem.lemma import LemmaReplacer
lemmatizer = LemmaReplacer('latin')
sentence = "Commius autem sive expiato suo dolore sive magna parte amissa suorum legatos ad Antonium mittit seque et ibi futurum, ubi praescripserit, et ea facturum, quae imperarit, obsidibus firmat;"  # Caesar 8.48.8
lemmatizer.lemmatize(sentence)

Output

File "/home/thibault/dev/latin-topic-recognition/venv/lib/python3.4/site-packages/cltk/stem/lemma.py", line 72, in lemmatize
    headword = self.lemmata[token.lower()]
KeyError: 'antonium'

Expected Output and Discussion

I am not expecting the lemmatizer to raise KeyError on unknown words/forms. Worst case, I would expect an option at the instantiation (LemmaReplacer(raise_on_missing=False)) which would output either the original form, a stemmed form or “unknown” (I obviously prefer the original form).

Would it be possible I am doing something wrong ?

[PonteIneptique]

Thanks a lot. I also think that a lemmatizer stack is a great idea. Mostly because we could put probabilistic one first (with fewer forms registered generally) and lookup one after 😃 Gj

[diyclassics]

@PonteIneptique You can see some preliminary work and testing here: https://github.com/diyclassics/cltk/tree/gsoc/cltk/lemmatize