"No handler for type [string] declared on field [test]" after update to ES6

Hi!

After updating from Elasticsearch 5.5.1 -> 6.1.1 (and corresponding elasticsearch-py and elasticsearch-dsl-py 5.x -> 6.x updates; running under Python 2.7) I get:

...
  File "/home/vagrant/treeherder/treeherder/model/search.py", line 197, in _init
    item.init()
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch_dsl/document.py", line 152, in init
    cls._doc_type.init(index, using)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch_dsl/document.py", line 85, in init
    self.mapping.save(index or self.index, using=using or self.using)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch_dsl/mapping.py", line 116, in save
    return index.save()
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch_dsl/index.py", line 219, in save
    return self.create()
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch_dsl/index.py", line 203, in create
    self.connection.indices.create(index=self._name, body=self.to_dict(), **kwargs)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 91, in create
    params=params, body=body)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 314, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 161, in perform_request
    self._raise_error(response.status, raw_data)
  File "/home/vagrant/python/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'mapper_parsing_exception', u'No handler for type [string] declared on field [test]')

Enabling elasticsearch.trace logging I see the request being made was:

[2018-01-02 14:38:52,749] INFO [elasticsearch.trace:67] curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/test-test-failure-line?pretty' -d '{
  "mappings": {
    "doc": {
      "properties": {
        "best_classification": {
          "index": "not_analyzed",
          "type": "integer"
        },
        "best_is_verified": {
          "index": "not_analyzed",
          "type": "boolean"
        },
        "expected": {
          "index": "not_analyzed",
          "type": "string"
        },
        "job_guid": {
          "index": "not_analyzed",
          "type": "string"
        },
        "message": {
          "analyzer": "message_analyzer",
          "type": "string"
        },
        "status": {
          "index": "not_analyzed",
          "type": "string"
        },
        "subtest": {
          "index": "not_analyzed",
          "type": "string"
        },
        "test": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "message_analyzer": {
          "filters": [],
          "tokenizer": "message_tokenizer",
          "type": "custom"
        }
      },
      "tokenizer": {
        "message_tokenizer": {
          "pattern": "0x[0-9a-fA-F]+|[\\W0-9]+?",
          "type": "pattern"
        }
      }
    }
  }
}'
[2018-01-02 14:38:52,749] DEBUG [elasticsearch.trace:70] #[400] (0.016s)
#{
#  "error": {
#    "caused_by": {
#      "reason": "No handler for type [string] declared on field [test]",
#      "type": "mapper_parsing_exception"
#    },
#    "reason": "Failed to parse mapping [doc]: No handler for type [string] declared on field [test]",
#    "root_cause": [
#      {
#        "reason": "No handler for type [string] declared on field [test]",
#        "type": "mapper_parsing_exception"
#      }
#    ],
#    "type": "mapper_parsing_exception"
#  },
#  "status": 400
#}

I believe this is coming from the usage of the field type String, here: https://github.com/mozilla/treeherder/blob/2adb7b93dc9148eda66998e85a0ca20d1e986ecd/treeherder/model/search.py#L68-L75

Looking at the changelog for elasticsearch-dsl 5.0.0, I see: String field type has been deprecated in favor of Text and Keyword

…so presumably this is related to that?

However if Elasticsearch 6 no longer supports string, then:

1. it seems like it should be removed from elasticsearch-dsl to prevent confusion / reduce the time spent debugging this? (Or at least a visible deprecation message logged somewhere?)
2. it would be great for the elasticsearch-dsl 6.0 release notes to mention it’s now changed from deprecated to “doesn’t work”

After figuring that out, I tried importing/using Text instead of String, however I then get: elasticsearch.exceptions.RequestError: TransportError(400, u'mapper_parsing_exception', u'Failed to parse mapping [doc]: Could not convert [test.index] to boolean')

Both of these feel like things that perhaps would be best caught in the client to ease debugging?

Many thanks 😃

[honzakral]

Hi, indeed it is caused by that - string is deprecated and you should be using text or keyword instead.

string can still, however, exist in a mapping in a 6.x cluster if the index was created before the upgrade, that is why it is still present in the library. We will most likely remove it soon though in order to avoid similar confusion, thank you for the report!

Hope this helps!

[willtu]

So how to solve the problem occurs after changing from “string” to “text”?

elasticsearch.exceptions.RequestError: TransportError(400, u’mapper_parsing_exception’, u’Failed to parse mapping [doc]: Could not convert [test.index] to boolean’)