how can we add Stanford CoreNLU Arabic language support to Rasa-core? or is there any other way to add support for arabic language?

Rasa NLU version: master branch

Operating system (windows, osx, …): windows

Content of model configuration file:

Issue:

[91ns]

@reza-ebrahimi

If you pull the latest version of Rasa NLU and use this pipeline it will work for Arabic Intent Classification + NER. I tried this configuration and it’s very good.

language: “ar”

pipeline:

• name: “tokenizer_whitespace”
• name: “ner_crf”
• name: “intent_featurizer_count_vectors”
• name: “intent_classifier_tensorflow_embedding” intent_tokenization_flag: true intent_split_symbol: “_”

[andreasstuber]

@akelad , we just tested the latest PR with Arabic, bot for intent and entity recognition. Seems to work just fine with a small test (not yet representative). We’ll do some more exhaustive testing with some other languages (Urdu, Jeriza, Yoruba) and larger samples. This is really exciting!