Non deterministic results

In order to test the process of disambiguation, some possibilities of query were given. Let’s take test cases with disambiguation of Pdf files, the service showed a strange behavior since it gave different results even for the same query. The following is some test cases done on a Pdf file with the same query template: 2009.Infiniti.pdf

{
    "mentions": [
        "ner",
        "wikipedia"
    ],
    "nbest": false,
    "customisation": "generic"
}

The service gave different results, for instance the mention ‘Francesco Speranza’ can be full recognized as ‘Francesco Speranza’, can partially recognized as ‘Speranza’, or even cannot be recognized at all.

Below are some screenshots of the results.

1. ‘Francesco Speranza’ can be full recognized

2. ‘Francesco Speranza’ can partially recognized as ‘Speranza’

3. ‘Francesco Speranza’ cannot be recognized at all

[kermitt2]

Also note that it has nothing specific to the PDF, it is apparent whatever input we provide to the monster 😃

[kermitt2]

This is something visible for long. My hypothesis is that it is due to the random seed of SMILE, leading to different partitioning of decision trees in the ensemble decision algo, and thus different probabilities - above or bellow the thresholds, and finally to this non-deterministic behaviour. If it comes from that, in sklearn we typically fix the random seed for reproducibility.