Ping fails if user doesn't have cluster monitor permission

NEST/Elasticsearch.Net version: 6.8.0

Elasticsearch version: 7.1.1

Description of the problem including expected versus actual behavior:

I have a cluster with xpack security enabled, and a low privilege “write only” user whose roles only allow create_index and index for specific index patterns.

When I use ElasticSearch.net with this user (via NLog.Targets.ElasticSearch), configured with a static connection pool containing two nodes, operations fail with the following exception.

ElasticSearch: Failed to send log messages. status=403 Exception: Elasticsearch.Net.ElasticsearchClientException: Maximum number of retries reached, failed over to all the known alive nodes before failing. Call: Status code 403 from: HEAD / ---> System.AggregateException: One or more errors occurred. ---> Elasticsearch.Net.PipelineException: Failed to ping the specified node. ---> Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.GetResponse()
   at Elasticsearch.Net.HttpWebRequestConnection.Request[TResponse](RequestData requestData)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.Ping(Node node)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.Ping(Node node)
   at Elasticsearch.Net.Transport`1.Ping(IRequestPipeline pipeline, Node node)
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)

Presumably because of the following (the ElasticSearch.net client looks like it attempts to hit the / page on the cluster node to determine whether it’s up):

# curl -u testuser:password -XGET "http://xxxx:9200/"
{"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [testuser]"}],"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [testuser]"},"status":403}

# curl -u testuser:password -XGET "http://xxxx:9200/" --head
HTTP/1.1 403 Forbidden
content-type: application/json; charset=UTF-8
content-length: 243

This can be worked around by either disabling pinging, or giving the low privilege user the cluster:monitor permission.

While maybe not a bug per-se, is this the expected behaviour? Could the ping operation be modified to do something that doesn’t require cluster monitoring permission? Or is disabling ping in this situation the correct thing to do?

Steps to reproduce:

1. Create an elasticsearch user without cluster monitoring permissions.
2. Connect to the elasticsearch cluster with pinging enabled.
3. Try and perform an operation, ping fails due to request to / returning a 403.

[russcam]

NEST/Elasticsearch.Net version: 6.8.0

Elasticsearch version: 7.1.1

An aside: You should use NEST 7.x with Elasticsearch 7.x. NEST 6.x is not compatible or tested with Elasticsearch 7.x.

A StaticConnectionPool tries to ping when a node in the pool is first used. A security exception is expected behaviour, if the user making the ping does not have permissions to make a HEAD request against /.

This can be worked around by either disabling pinging, or giving the low privilege user the cluster:monitor permission.

These are the approaches to take. You can limit the permission further to "cluster:monitor/main".

[c4m4l340]

That is strange, because, the user associated with the key, has the permission cluster:monitor/main, the key was generated after the user had the permission, so the apikey should have the permission also.