Evaluating Neural Network model using Errant

Dear @all

I have an overview of your documentation, I’m still confused about how to evaluate my Neural Network model (GEC). As I understood that, I have to translate the test set (correcting), then build a new (M2) file using errant_parallel command. The last step is to use errant_compare with the span-based correction to get F0.5 score.

Is this correct? What is the optimal way to evaluate my NN model using Errant?

Regards,

[chrisjbryant]

Looks like you forgot to install a spacy model.

You need to run python3 -m spacy download en after you pip install errant. It’s the last line in the installation instructions.

[chrisjbryant]

Hi,

You don’t need to be confused because you are correct!

1. Use errant_parallel to build a new M2 file from the original sentences and your corrected sentences.
2. Use errant_compare to compare your system M2 file with the gold standard M2 file.

That’s all there is to it. 😃