Port number automatically assigned when not needed

Hello!

I had an issue using elasticsearch-py to connect to an elasticsearch index that was resulting from the url formatting in connection.http_requests.RequestHttpConnection. In the __init__ of this class, a default value for port is specified which was making its way into the url which seemed to allow the connection but caused a TransportError(404, u'index_not_found_exception', u'no such index') when trying to search my index. Here’s the code I used:

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth

host = "my_elasticsearch_index.es.amazonaws.com/"
AWS_ACCESS_KEY_ID = "the_access_key"
AWS_SECRET_ACCESS_KEY = "the_secret_access_key"
AWS_REGION = "my-region"

awsauth = AWS4Auth(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, 'es')

es = Elasticsearch(
    hosts=[{'host': host}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)
query =  {}
page = es.search(
    index="my-index-name",
    scroll='2m',
    size=1000,
    body=query
)

The error occurs on the page=es.search() line:

  File "/venv/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
NotFoundError: TransportError(404, u'index_not_found_exception', u'no such index')

Removing the port value specification fixed the issue. The default port number was making the url come out as https://my_elasticsearch_index.es.amazonaws.com/:9200/my-index-name I ended up making the following modification to the __init__ in RequestHttpConnection around line 46:

        if port is not None and type(port) == int:
            self.base_url = 'http%s://%s:%d%s' % (
                's' if self.use_ssl else '',
                host, port, self.url_prefix
            )
        elif port is None:
            self.base_url = 'http%s://%s%s' % (
                's' if self.use_ssl else '',
                host, self.url_prefix
            )

And changed the code for the variable es to specify the port as None:

es = Elasticsearch(
    hosts=[{'host': host, 'port': None}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)

This works, and formats the url properly, but I’m wondering: is there another methodology I could use to connect to AWS and query an index without modifying elasticsearch-py?

[fxdgear]

By default Elasticsearch is available on port 9200. Which is why elasticsearch-py behaves this way. If you want to use a different port you can specify any port you want.

It looks like you are using Elasticsearch running on port 443. (unless you’re using http, then it would be port 80).

Default ports:

• http://blah -> port 80
• https://blah -> port 443

I would recommend not altering the elasticsearch-py code and instead instantiate your connection using the port that your elasticsearch instance is running on.

es = Elasticsearch(
    hosts=[{'host': host, 'port': 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)

I’m making an assumption on port 443 here since you have use_ssl=True

[mr-wolverine]

I have same problem. Why this issues is closed? Is there any solutions? I didn’t find one from the context.