[PROPOSAL] Introduce `spring-data-opensearch` to support OpenSearch integration

Picking up the discussion started in https://github.com/spring-projects/spring-data-elasticsearch/issues/1770, at this moment it becomes clear that both projects are going to evolve in (slightly) incompatible fashion. Taking into account how widely popular Spring Data Elasticsearch is, it is highly likely that community would seek the first class support of the OpenSearch integration as well.

In scope of this issue, it would be great to hear the feedback on the proposal to introduce dedicated spring-data-opensearch project so to have OpenSearch supported out of the box, not depending on Elasticsearch. @sothawo if there is an interest and willing to accept the contribution, I would be more than happy to work on the pull request.

The OpenSearch project just published the Maven artifacts for client libraries [1], there should be no issues to rely on those for spring-data-opensearch and disconnect from Elasticsearch.

Thank you very much.

[1] https://discuss.opendistrocommunity.dev/t/maven-repository-artifacts-for-plugin-development/6406/7]

[sothawo]

Yesterday we had a call - @brijos (AWS/Opensearch), @mp911de and Ilayaperumal Gopinathan (both VMware/Spring Data) @reta and me - about how and where the integration of Opensearch into Spring Data Elasticsearch will be done. The result of this call:

• There will be a new artifact spring-data-opensearch.
• This will not be a modified clone of the existing Spring Data Elasticsearch library, but will be built on top of that, providing the integration of the Opensearch client into Spring Data Elasticsearch.
• The repository for that new library will be set up and maintained by Opensearch and the community
• The code of the PR from @reta will be the initial code for this integration

The issue for this setup in Opensearch: https://github.com/opensearch-project/opensearch-clients/issues/28.

This setup allows changes in Spring Data Elasticsearch to be available for users of Opensearch as well as long as no client modifications are necessary. Once this setup is done, the documentation in Spring Data Elasticsearch will be adapted to inform the users about the new possibility to integrate Opensearch.

[xeraa]

Only solution is to go OpenSearch now Elastic has sabotaged the Spring Data Elastic Search project by making it unusable for most cases. Elastic has tainted the transitive dependencies of Spring Data Elastic Search with strong copy-left licenses that make any project and code using Spring Data Elastic Search become itself OSS.

No. The Elasticsearch clients are forward compatible — if you use the client for Elasticsearch 7.15, it expects the full 7.15 API (or newer); not an older version which misses endpoints or a different API. This has been causing more than enough issues and complaints in the past that we are clear about it from starting rather than having runtime issues later on. It just doesn’t work.

If you use an old version of Elasticsearch, use that version of the client. You’re not gaining anything from upgrading just the client. If you’re using a different project with a diverging API, you’ll need a client for that.

I do love OSS but I love getting paid even more. So I need to develop proprietary software for my employers. I try to contribute to OSS when ever I can but the code I get paid for unfortunately needs to be proprietary.

I really liked the Elastic Search, but unfortunately the Elastic company has become toxic and they started sabotaging or booby-trapping all their software.

We also like getting paid. We actually give most of our work away for free and still do. But others monetizing it (instead of us) has reached a point where it wasn’t reasonable any more. Given your opening sentence here, I appreciate your understanding 😃

Spring Data Elastic Search is now tainted from Elastic with

1. Server Side Public License, which is strong copy-left license
2. Elastic License which forbids commercial use and retail of your code.

No. We waived it for the now deprecated HLRC client and the new Java API client is Apache 2 licensed as announced last year.