Error: binaryEmbeddingReader can't be null

I’m using Elasticsearch as docker container with the binary-vector-scoring plugin installed, but I’m getting an intermittent error when doing search with the following query:

{
  "function_score": {
    "boost": 1,
    "score_mode": "avg",
    "boost_mode": "multiply",
    "min_score": 0,
    "script_score": {
      "script": {
        "source": "binary_vector_score",
        "lang": "knn",
        "params": {
          "cosine": true,
          "field": "image_embedding",
          "vector": "MY_VECTOR_HERE"
        }
      }
    }
  }
}

The search runs ok for a while (first dozen of requests) and then it starts returning the following error:

Caused by: java.lang.IllegalStateException: binaryEmbeddingReader can't be null
elasticsearch    | 	at com.liorkn.elasticsearch.script.VectorScoreScript.setBinaryEmbeddingReader(VectorScoreScript.java:67) ~[?:?]
elasticsearch    | 	at com.liorkn.elasticsearch.service.VectorScoringScriptEngineService$1.getLeafSearchScript(VectorScoringScriptEngineService.java:65) ~[?:?]
elasticsearch    | 	at org.elasticsearch.common.lucene.search.function.ScriptScoreFunction.getLeafScoreFunction(ScriptScoreFunction.java:79) ~[elasticsearch-5.6.0.jar:5.6.0]
elasticsearch    | 	at org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$CustomBoostFactorWeight.functionScorer(FunctionScoreQuery.java:140) ~[elasticsearch-5.6.0.jar:5.6.0]
...

Reindexing all documents is the only way to make the search work again, has anybody faced the same problem?

[lior-k]

please do the following query in order to check that all the documents have values in this field. meaning this query should return 0 documents:

GET <es-url>/<index>/_search
{
    "query": {
        "bool" : {
            "must" : {
                "script" : {
                    "script" : {
                        "inline": "doc.image_embedding == null || doc.image_embedding.value == null || doc.image_embedding.value == ''",
                        "lang": "painless"
                     }
                }
            }
        }
    }
}

[tgreiser]

Also struggling with this problem. The plugin works in production, but when I use elasticdump to copy the data to a local server I start getting “binaryEmbeddingReader can’t be null”.

elasticdump --input=./account_mapping.json --output=http://localhost:9200/account --type=mapping
elasticdump --input=./account.json --output=http://localhost:9200/account --type=data

In this state my vector searches fail entirely. If I inspect the mapping my field is mapped correctly. If I use the painless query above I find 0 records. If I reindex my document then things start working on most of the shards.

POST http://localhost:9200/_reindex
{
  "source": {
    "index": "account"
  },
  "dest": {
    "index": "tmp"
  }
}

Then I do a second _reindex to rename from tmp back to account. My queries start working now, however - I still see exceptions firing in the ES server and my query _shards has 3 successful and 2 failed shards:

"_shards": {
        "total": 5,
        "successful": 3,
        "skipped": 0,
        "failed": 2,
        "failures": [
            {
                "shard": 0,
                "index": "account",
                "node": "HlfEVuX_TbO8u6GXu47REQ",
                "reason": {
                    "type": "illegal_state_exception",
                    "reason": "binaryEmbeddingReader can't be null"
                }
            }
        ]
    },

Update: After about 15 minutes and a few reboots, the two buggy shards started working and I am getting 5/5 successful now. So if anyone else has the same problem - import, reindex and then wait a while while shards rebuild.