[meta] 6.5.0 Release

This is a meta issue to keep track of the work we want to do for the 6.4.0 release individual tickets with the v6.4.0 tag will be created for most of these.

Feature parity work

Deprecations

New Features

6.4.0 leftover work

• Add Kerberos authentication support - https://github.com/elastic/elasticsearch/pull/32263
• ML. Detectors now support custom rules that enable the user to improve machine learning results by providing some domain-specific knowledge in the form of rule. - https://github.com/elastic/elasticsearch/issues/31110
• ML. Reverse engineer Grok patterns from categorization results - https://github.com/elastic/elasticsearch/pull/30125
• Add a format option to docvalue_fields. - https://github.com/elastic/elasticsearch/pull/29639

Enhancements

6.4.0 leftover work

• Return statistics about forecasts as part of the job stats and usage API - https://github.com/elastic/elasticsearch/pull/31647
• [ML] Add description to ML filters - https://github.com/elastic/elasticsearch/pull/31330
• Rename ranking evaluation quality_level to metric_score - https://github.com/elastic/elasticsearch/pull/32168
• Rename ranking evaluation response unknown_docs section - https://github.com/elastic/elasticsearch/pull/32166
• Add Expected Reciprocal Rank metric - https://github.com/elastic/elasticsearch/pull/31891
• Add details section for dcg ranking metric - https://github.com/elastic/elasticsearch/pull/31177

Bug fixes

New API’s

6.4.0 leftover work

• xpack.license.get_basic_status.json
• xpack.license.post_start_basic.json
• xpack.ml.delete_calendar.json
• xpack.ml.delete_calendar_event.json
• xpack.ml.delete_calendar_job.json
• xpack.ml.get_calendar_events.json
• xpack.ml.get_calendars.json
• xpack.ml.info.json
• xpack.ml.post_calendar_events.json
• xpack.ml.put_calendar.json
• xpack.ml.put_calendar_job.json
• xpack.ml.get_calendar_job.json
• xpack.ssl.certificates.json
• xpack.ml.update_filter.json
• xpack.rollup.get_rollup_index_caps.json
• xpack.security.delete_privileges.json
• xpack.security.get_privileges.json
• xpack.security.has_privileges.json
• xpack.security.put_privilege.json
• xpack.security.put_privileges.json
• nodes.reload_secure_settings.json

6.5.0 new API

• delete_by_query_rethrottle.json
• update_by_query_rethrottle.json
• xpack.ml.delete_forecast.json
• xpack.ml.find.file_structure.json
• xpack.security.get_user_privileges.json
• ccr.delete_auto_follow_pattern.json
• ccr.follow.json
• ccr.follow_stats.json
• ccr.get_auto_follow_pattern.json
• ccr.pause_follow.json
• ccr.put_auto_follow_pattern.json
• ccr.resume_follow.json
• ccr.stats.json
• ccr.unfollow.json

From the release notes:

• Handles exists query in composite aggs elastic/elasticsearch#35758
• [Scripting] Use Number as a return value for BucketAggregationScript elastic/elasticsearch#35653
• Require combine and reduce scripts in scripted metrics aggregation elastic/elasticsearch#33452
• Disallow “enabled” attribute change for types in mapping update (elastic/elasticsearch#33566) elastic/elasticsearch#33933
• Introduce index store plugins elastic/elasticsearch#32375
• Deprecates the standard filter elastic/elasticsearch#33468
• Deprecate negative query boost (elastic/elasticsearch#34486) elastic/elasticsearch#34512
• Deprecate negative weight in Function Score Query elastic/elasticsearch#33624
• Switch WritePipelineResponse to AcknowledgedResponse elastic/elasticsearch#32722
• Adds a new auto-interval date histogram elastic/elasticsearch#28993
• Add script_filter tokenfilter elastic/elasticsearch#33431
• Add conditional token filter to elasticsearch elastic/elasticsearch#31958
• Add get-user-privileges API elastic/elasticsearch#33928
• Add support for “authorization_realms” elastic/elasticsearch#33262
• [CCR] Added auto follow patterns feature elastic/elasticsearch#33118
• Rollup adding support for date field metrics (elastic/elasticsearch#34185) elastic/elasticsearch#34200
• Add early termination support for min/max aggregations elastic/elasticsearch#33375
• Add early termination support to BucketCollector elastic/elasticsearch#33279
• Add interval response parameter to AutoDateInterval histogram elastic/elasticsearch#33254
• Token API supports the client_credentials grant elastic/elasticsearch#33106
• Improved IndexNotFoundException’s default error message elastic/elasticsearch#34649
• Introduce index settings version elastic/elasticsearch#34429
• Introduce mapping version to index metadata elastic/elasticsearch#33147
• ingest: processor stats elastic/elasticsearch#34724
• ingest: better support for conditionals with simulate?verbose elastic/elasticsearch#34155
• ingest: correctly measure chained pipeline stats elastic/elasticsearch#33912
• ingest: support simulate with verbose for pipeline processor elastic/elasticsearch#33839
• ingest: Introduce the dissect processor elastic/elasticsearch#32884
• [ingest] geo-ip performance improvements elastic/elasticsearch#33029
• INGEST: Add Pipeline Processor elastic/elasticsearch#32473
• Ingest: Add conditional per processor elastic/elasticsearch#32398
• Introduce the dissect library elastic/elasticsearch#32297
• INGEST: Enable default pipelines elastic/elasticsearch#32286
• INGEST: Implement Drop Processor elastic/elasticsearch#32278
• Ingest: Support integer and long hex values in convert elastic/elasticsearch#32213
• Add region ISO code to GeoIP Ingest plugin elastic/elasticsearch#31669
• [ML] Add an ingest pipeline definition to structure finder elastic/elasticsearch#34350
• [ML] Add a timeout option to file structure finder elastic/elasticsearch#34117
• [ML] Allow asynchronous job deletion elastic/elasticsearch#34058
• Make certain ML node settings dynamic (elastic/elasticsearch#33565) elastic/elasticsearch#33961
• Adding node_count to ML Usage (elastic/elasticsearch#33850) elastic/elasticsearch#33863
• [ML] Minor improvements to categorization Grok pattern creation elastic/elasticsearch#33353
• Preserve the order of nested documents in the Lucene index elastic/elasticsearch#34225
• [ML] Delete forecast API (elastic/elasticsearch#31134) elastic/elasticsearch#33218
• Use the global doc id to generate random scores elastic/elasticsearch#33599
• Add minimal sanity checks to custom/scripted similarities. (backport) elastic/elasticsearch#33893
• Propagate auto_id_timestamp in primary-replica resync elastic/elasticsearch#33964
• [Rollup] Add support for date histo format when searching elastic/elasticsearch#34537
• [Rollup] Only allow aggregating on multiples of configured interval elastic/elasticsearch#32052
• Introduce a search_throttled threadpool elastic/elasticsearch#33732
• Expose max_concurrent_shard_requests in _msearch elastic/elasticsearch#33016
• Add a limit for graph phrase query expansion elastic/elasticsearch#34031
• Introduce private settings elastic/elasticsearch#33327
• Add user-defined cluster metadata elastic/elasticsearch#33325
• Add _source-only snapshot repository elastic/elasticsearch#32844
• Make XContentBuilder in AliasActions build is_write_index field elastic/elasticsearch#35071

Netcoreapp2.1

or 2.2 whichever comes first 😸.

• Explicitly ship this TFM and get some numbers how the managed handler performs on linux/windows through Tests.Throughput vs the already existing implementations/
• Move our dotnet core IHttpConnection over to HttpClientFactory. We currently do much of the same but if we don’t have to maintain our own pooling of HttpClient instances that would eleviate some complexity in our codebase.
• Span based JSON serializer implementation?

Elasticsearch.Net.Abstractions

• Use differ from this repository see: https://github.com/elastic/elasticsearch-net-abstractions/pull/13
• Move ApiGeneration over.
• Move DocGeneration over.

[Mpdreamz]

Diff.zip

Attached is the output for AssemblyDiffer for NEST and Elasticsearch.net against 6.4.2 as released on Nuget.

LGTM could use extra pair of eyes to validate @russcam @codebrain

[codebrain]

@Mpdreamz - checked both XMLs, they look good.