MapperParsingException while running Rally against 1.4.4 single node cluster with --pipeline=benchmark-only

Rally version (get with esrally --version): 0.4.8

Invoked command: esrally --target-hosts=<server>:9200 --pipeline=benchmark-only

Configuration file (located in ~/.rally/rally.ini)):

[meta] config.version = 6

[system] root.dir = /Users/claudej/.rally/benchmarks log.root.dir = logs env.name = local

[source] local.src.dir = /Users/claudej/.rally/benchmarks/src remote.repo.url = https://github.com/elastic/elasticsearch.git

[build] gradle.bin = /usr/local/bin/gradle

[provisioning] local.install.dir = install

[runtime] java8.home = /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home

[benchmarks] local.dataset.cache = ${system:root.dir}/data

[reporting] datastore.type = in-memory datastore.host = datastore.port = datastore.secure = datastore.user = datastore.password =

[tracks] default.url = https://github.com/elastic/rally-tracks

[defaults] preserve_benchmark_candidate = False

JVM version: 1.8.0_77 (Rally); 1.8.0_60 (ES server)

OS version: MacOS 10.11.6 (Rally); CentOS 6.8 (ES server)

Description of the problem including expected versus actual behavior:

Unable to start benchmark run. Fails after 2 seconds but the index is created:

claudej35-mac:22-Feb-2017 claudej$ curl -k -XGET http://<server>:9200/_cat/indices?pretty red open geonames 5 0

claudej35-mac:22-Feb-2017 claudej$ curl -k -XGET http://<server>:9200/_cat/shards geonames 4 p UNASSIGNED
geonames 0 p UNASSIGNED
geonames 3 p UNASSIGNED
geonames 1 p UNASSIGNED
geonames 2 p UNASSIGNED

I have confirmed I can talk to the ES version using curl(1) so there’s no firewalling or networking problem.

Steps to reproduce: 1. 2. 3.

Provide logs (if relevant):

Log output (truncated):

. . 2017-02-23 01:29:08,979 rally.client INFO Creating ES client connected to [{‘host’: ‘<server>’, ‘port’: ‘9200’}] with options [{‘timeout’: 60000, ‘request_timeout’: 60000}] 2017-02-23 01:29:09,121 rally.launcher INFO Distribution version was not specified by user. Rally-determined version is [1.4.4] . . 2017-02-23 01:29:10,151 rally.driver INFO Benchmark is about to start. 2017-02-23 01:29:10,151 rally.client INFO Creating ES client connected to [{‘host’: ‘<server>’, ‘port’: ‘9200’}] with options [{‘timeout’: 60000, ‘request_timeout’: 60000}] 2017-02-23 01:29:10,152 rally.metrics INFO Opening metrics store for invocation=[20170223T012908Z], track=[geonames], challenge=[append-no-conflicts], car=[external] 2017-02-23 01:29:10,701 elasticsearch WARNING HEAD /geonames [status:404 request:0.548s] 2017-02-23 01:29:10,701 rally.driver INFO Creating index [geonames] 2017-02-23 01:29:10,817 rally.driver INFO create mapping for type [type] in index [geonames] with content: { “type”: { “dynamic”: “strict”, “properties”: { “elevation”: { “type”: “integer” }, “name”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “geonameid”: { “type”: “long” }, “feature_class”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “longitude”: { “type”: “double” }, “cc2”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “timezone”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “dem”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “country_code”: { “type”: “text”, “fielddata”: true, “fields”: { “raw”: { “type”: “keyword” } } }, “admin1_code”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “admin2_code”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “admin3_code”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “admin4_code”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “latitude”: { “type”: “double” }, “feature_code”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “alternatenames”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “asciiname”: { “type”: “text”, “fields”: { “raw”: { “type”: “keyword” } } }, “population”: { “type”: “long” } } } } 2017-02-23 01:29:10,925 elasticsearch WARNING PUT /geonames/_mapping/type [status:400 request:0.108s] 2017-02-23 01:29:10,926 rally.driver ERROR Main driver encountered a fatal exception. Shutting down. Traceback (most recent call last): File “/usr/local/lib/python3.6/site-packages/esrally/driver/driver.py”, line 140, in receiveMessage self.start_benchmark(msg, sender) File “/usr/local/lib/python3.6/site-packages/esrally/driver/driver.py”, line 190, in start_benchmark setup_index(self.es, index, self.challenge.index_settings) File “/usr/local/lib/python3.6/site-packages/esrally/driver/driver.py”, line 503, in setup_index body=json.loads(mappings)) File “/usr/local/lib/python3.6/site-packages/elasticsearch/client/utils.py”, line 69, in _wrapped return func(*args, params=params, **kwargs) File “/usr/local/lib/python3.6/site-packages/elasticsearch/client/indices.py”, line 291, in put_mapping ‘_mapping’, doc_type), params=params, body=body) File “/usr/local/lib/python3.6/site-packages/elasticsearch/transport.py”, line 329, in perform_request status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) File “/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py”, line 109, in perform_request self._raise_error(response.status, raw_data) File “/usr/local/lib/python3.6/site-packages/elasticsearch/connection/base.py”, line 108, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) elasticsearch.exceptions.RequestError: <unprintable RequestError object> 2017-02-23 01:29:10,930 root ERROR Cannot run subcommand [race]. Traceback (most recent call last): File “/usr/local/lib/python3.6/site-packages/esrally/rally.py”, line 459, in dispatch_sub_command racecontrol.run(cfg) File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 237, in run raise e File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 234, in run pipeline(cfg) File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 48, in call self.target(cfg) File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 182, in benchmark_only return race(Benchmark(cfg, mechanic.create(cfg, metrics_store, external=True), metrics_store), cfg) File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 156, in race benchmark.run(lap) File “/usr/local/lib/python3.6/site-packages/esrally/racecontrol.py”, line 80, in run raise exceptions.RallyError(result.message, result.cause) esrally.exceptions.RallyError: (‘Could not execute benchmark’, RequestError(400, ‘MapperParsingException[No handler for type [text] declared on field [name]]’, {‘error’: ‘MapperParsingException[No handler for type [text] declared on field [name]]’, ‘status’: 400})) 2017-02-23 01:29:10,932 rally.main INFO Attempting to shutdown internal actor system. 2017-02-23 01:29:11,15 rally.main INFO Shutdown completed.

[danielmitterdorfer]

Thanks for the feedback @mhohara! I just fixed the underlying problem (see #242) which will be included in Rally 0.5.3. Then the workaround will not necessary anymore.

[khyron4eva]

@danielmitterdorfer,

My initial tests were across a WAN (laptop to server). Later tests have been within the same datacenter but I doubt they have spatial locality at the rack level. But even just running in the same datacenter has resolved any of the previous issues.

Thanks!