Can't access only authorized index

I have access to only specific indexes in an ES cluster. So when I do

es = Elasticsearch("https://<server>:port", http_auth=('<user>','<passwd>'))

This raises an exception:- elasticsearch.exceptions.AuthorizationException: TransportError(403, 'security_exception', 'action [cluster:monitor/main] is unauthorized for user [<user>]')

So I tried to suffix the url with my specific-index

es = Elasticsearch("https://<server>:port/<specific-index>", http_auth=('<user>','<passwd>'))

This works and returns me information about the index. But then I’m not able to do any queries on it since the url is translated to https://<server>:port/<specific-index>/<specific-index>/<doc-type>/<id>

Is there a way for this to be like lazy evaluated? (I mean, the es object is created but not initialized until I run the query on my specific-index?)

[fxdgear]

@dibyadas I’m sorry for the delay but been busy with other things then the weekend happened and I had to do real life. 😦

Anyway I’ve done some digging and realized that this is the expected behavior.

What I’ve done is I created a new cluster with x-pack trial, this cluster has the following:

• index called regularindex
• role called regularrole
• user called regularuser with password regularuser.
• the role regularrole is only able to view the index regularindex.

curl -k "https://elastic:changeme@localhost:9200/_xpack/security/role/regularrole?pretty"
{
  "regularrole" : {
    "cluster" : [ ],
    "indices" : [
      {
        "names" : [
          "regularindex"
        ],
        "privileges" : [
          "all"
        ],
        "field_security" : {
          "grant" : [
            "*"
          ]
        }
      }
    ],
    "run_as" : [ ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}

Now if we use curl to log into the cluster with our regular user and just view the root url / we get this error:

curl -k "https://regularuser:regularuser@localhost:9200?pretty"
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [cluster:monitor/main] is unauthorized for user [regularuser]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [cluster:monitor/main] is unauthorized for user [regularuser]"
  },
  "status" : 403
}

BUT if we look our our index regularindex you can see that the request is authorized:

curl -k "https://regularuser:regularuser@localhost:9200/regularindex?pretty"
{
  "regularindex" : {
    "aliases" : { },
    "mappings" : { },
    "settings" : {
      "index" : {
        "creation_date" : "1530558683107",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "DyR3EguMQlODErb4GTw00g",
        "version" : {
          "created" : "6030099"
        },
        "provided_name" : "regularindex"
      }
    }
  }
}

Translating this into the python client we get the same behavior:

In [1]: from elasticsearch import Elasticsearch

In [2]: es = Elasticsearch('elasticsearch', use_ssl=True, verify_certs=False, http_auth=('regularuser','regularuser'))
/code/elasticsearch-py/elasticsearch/connection/http_urllib3.py:134: UserWarning: Connecting to elasticsearch using SSL with verify_certs=False is insecure.
  'Connecting to %s using SSL with verify_certs=False is insecure.' % host)

#### 
#### Here you can see that we get unauthorized error as expected.
####

In [3]: es.info()
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
GET https://elasticsearch:9200/ [status:403 request:0.036s]
---------------------------------------------------------------------------
AuthorizationException                    Traceback (most recent call last)
<ipython-input-3-f69669341f07> in <module>()
----> 1 es.info()

/code/elasticsearch-py/elasticsearch/client/utils.py in _wrapped(*args, **kwargs)
     74                 if p in kwargs:
     75                     params[p] = kwargs.pop(p)
---> 76             return func(*args, params=params, **kwargs)
     77         return _wrapped
     78     return _wrapper

/code/elasticsearch-py/elasticsearch/client/__init__.py in info(self, params)
    239         `<http://www.elastic.co/guide/>`_
    240         """
--> 241         return self.transport.perform_request('GET', '/', params=params)
    242
    243     @query_params('parent', 'pipeline', 'refresh', 'routing', 'timeout',

/code/elasticsearch-py/elasticsearch/transport.py in perform_request(self, method, url, headers, params, body)
    316                 delay = 2**attempt - 1
    317                 time.sleep(delay)
--> 318                 status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
    319
    320             except TransportError as e:

/code/elasticsearch-py/elasticsearch/connection/http_urllib3.py in perform_request(self, method, url, params, body, timeout, ignore, headers)
    183         if not (200 <= response.status < 300) and response.status not in ignore:
    184             self.log_request_fail(method, full_url, url, body, duration, response.status, raw_data)
--> 185             self._raise_error(response.status, raw_data)
    186
    187         self.log_request_success(method, full_url, url, body, response.status,

/code/elasticsearch-py/elasticsearch/connection/base.py in _raise_error(self, status_code, raw_data)
    122             logger.warning('Undecodable raw error response from server: %s', err)
    123
--> 124         raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
    125
    126

AuthorizationException: AuthorizationException(403, 'security_exception', 'action [cluster:monitor/main] is unauthorized for user [regularuser]')


#### 
#### Here you can see that we can get interact with our index as expected. 
####

In [4]: es.indices.get('regularindex')
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
Out[4]:
{'regularindex': {'aliases': {},
  'mappings': {},
  'settings': {'index': {'creation_date': '1530558683107',
    'number_of_shards': '5',
    'number_of_replicas': '1',
    'uuid': 'DyR3EguMQlODErb4GTw00g',
    'version': {'created': '6030099'},
    'provided_name': 'regularindex'}}}}

Long story short, if you want your user to be able to do es.info() you have to give permissions to your role to access monitoring.

[fxdgear]

@dibyadas ok great. thank you for the feedback. I wanted to be sure you were using a security plugin (aka x-pack) that I am familiar with, and try to reproduce the issue.