Making signed requests to Elasticsearch

Hi,

I’m writing a python lambda function to process incoming ELB logs and put them into an Elasticsearch domain. So far the boto3 library has been great, but I’m struggling to find out how to either make my bulk index post through the client or how to sign a direct rest api call.

I thought the generate_presigned_url call might be what I want, but I can’t find out what is a valid ClientMethod

Is there an easy way to sign an outgoing request through the boto3 library using assumed iam role credentials?

Thanks, Alex

[alexrudd]

Thanks!

For anyone else who comes across this problem, here’s what I ended up doing:

import boto3
from aws_requests_auth.aws_auth import AWSRequestsAuth
from elasticsearch import Elasticsearch, RequestsHttpConnection

session = boto3.session.Session()
credentials = session.get_credentials().get_frozen_credentials()

es_host = 'search-my-es-domain.eu-west-1.es.amazonaws.com'
awsauth = AWSRequestsAuth(
    aws_access_key=credentials.access_key,
    aws_secret_access_key=credentials.secret_key,
    aws_token=credentials.token,
    aws_host=es_host,
    aws_region=session.region_name,
    aws_service='es'
)

# use the requests connection_class and pass in our custom auth class
es = Elasticsearch(
    hosts=[{'host': es_host, 'port': 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)

print(es.info())

[tedder]

It was intentionally broken by Elastic: https://www.theregister.com/2021/08/09/elasticsearch_python_client_change/ https://github.com/elastic/elasticsearch-py/pull/1623