Integration of official async ES client into DSL library

Hi team,

Great effort on contributing and maintaining a great layer on top of ES access.

While there are some discussions around the async integration into the official DSL library like #704 and #556 I did not find any conclusion on that. I did my research and also found there isn’t anything existing. While there are some prototypes and suggestions like here I did not see an implementation.

I was presuming, it could be due to two reasons: Either its really not there and no body has raised a PR yet, (or) this could be a trivial implementation so most users took care by overriding the parent classes.

In any case, I needed the integration of the official async python library into DSL. I was experimenting many ways, and something simple yet required to duplicate was to create async_xyz classes on top of the low level APIs that were accessing the es client functions and eventually not awaiting the coroutines from async ES client.

That said, here is a very basic working/and tested implementation in my fork https://github.com/elastic/elasticsearch-dsl-py/compare/master...vaidsu:master

Please consider this as a basic prototype implementation and I just started writing tests. I am not sure, if this is good to go, or not. Then if yes, need to understand how far I need to write tests, I am just planning to make all the sync tests runnable on async calls.

Please let me know. If you like the thought, but have suggestions, also feel free to share – I am open to contribute back.

Thanks

[timothycrosley]

Now that Python2 is imminently reaching EOL, would a Python3 solution be accepted? If not, what would need to be changed from @vaidsu’s solution to make it acceptable?

Thanks!

~Timothy

[braunsonm]

Thanks for the info @vaidsu

Is there a way to access the async es wrapper after initializing the DSL with create_connection? If so that would be useful as well for using the regular elasticsearch py syntax as well.

from elasticsearch_dsl.connections import connections

es = connections.create_connection(["hosts"])  # Way to create the connection with the async wrapper?
hw = HelloWorld(
        hello='foo',
        timestamp=time.time())
await hw.async_save()
await hw.async_update(hello="anotherworld", timestamp=int(time.time() * 1000))

# Async update?
await es.update(body={
    "script": {
        ...
    }
}, index='myindex', doc_type='hello_world', id=hw.meta.id)

Since there isn’t a way to run update scripts in the DSL right now there are some points where the underlying API has to be used.

Seems like your implementation idea is well thought out though and I hope it can be merged in the near future 😃