Implementation issue

Dear Chris 😃

I have applied Errant for fce.test.me and I got unexpected results as :

=========== Span-Based Correction ============
TP      FP      FN      Prec    Rec     F0.5
2       15503   4547    0.0001  0.0004  0.0002
=======================================

I follow you implementation as same as in the documentation as follow:

1. I have applied errant_parallel using, errant_parallel -orig m2Scripts/orig_sentes.txt -cor m2Scripts/corec_sentes.txt -out m2Scripts/output.m2 files as in GitHub. Actually, I’m confused about the format of the parallel corrected text file.

2. For the errant_m2 I applied as errant_m2 -auto m2Scripts/output.m2 -out m2Scripts/auto_output.

3. The last step errant_compare as errant_compare -hyp m2Scripts/auto_output -ref m2Scripts/fce.test.m2 , results was are :

=========== Span-Based Correction ============
TP      FP      FN      Prec    Rec     F0.5
2       15503   4547    0.0001  0.0004  0.0002
=======================================

Could you please help to fix this issue?

Kind regards Aiman Solyman

[chrisjbryant]

I just ran errant with your files and got a similar score so can confirm errant is working properly.

The problem is your corec_sents.txt file. One obvious issue is that everything is now in lower case, but you also probably have a bug in your system since some of the output tokens are not real words (example in bold below)!

Orig:

As far as the organisation is concerned , an adequate number of double rooms with shower or bath have been booked for your group at the Palace Hotel , located across the main street from the train station .

Cor:

as far as organisation is concerned , adquire number of double rooms with shower or bath have been booked for your group at the paceal hotel located the main street from the traination stations .

Hopefully you can find the bug, but this is ultimately not an ERRANT issue!

[aimanmutasem]

Really, thank you so much for kindly support, 😃 All files are uploaded in Github, also uploaded notebook of implementation that showed BLEU-4: 79.517%.