[meta] 6.4.0 release

See original GitHub issue

This is a meta issue to keep track of the work we want to do for the 6.4.0 release individual tickets with the v6.4.0 tag will be created for most of these.

Feature parity work

Deprecations

Add deprecation warnings / changes for nGram and edgeNGram - https://github.com/elastic/elasticsearch/pull/30209
Investigate copy_settings default - https://github.com/elastic/elasticsearch/pull/30404
Default shards changing from 5 to 1 - https://github.com/elastic/elasticsearch/pull/30587
Deprecate not copy settings and explicitly disallow - https://github.com/elastic/elasticsearch/pull/30404 (https://github.com/elastic/elasticsearch-net/pull/3416)
Deprecate accepting malformed requests in stored script API - https://github.com/elastic/elasticsearch/pull/28939 Check, suspect we are OK
In the field capabilities API, deprecate support for providing fields in the request body. https://github.com/elastic/elasticsearch/pull/30157
Deprecate indexing and querying a context completion field without context - https://github.com/elastic/elasticsearch/pull/30712

New Features

Add WeightedAvg metric aggregation - https://github.com/elastic/elasticsearch/pull/31037 (https://github.com/elastic/elasticsearch-net/pull/3417)
Add a MovingFunction pipeline aggregation, deprecate MovingAvg - https://github.com/elastic/elasticsearch/pull/29594 (https://github.com/elastic/elasticsearch-net/pull/3419)
Add missing_bucket option in the composite agg - https://github.com/elastic/elasticsearch/pull/29465
Expose lucenes RemoveDuplicatesTokenFilter - https://github.com/elastic/elasticsearch/pull/31275
Multiplexing token filter - https://github.com/elastic/elasticsearch/pull/31208
Adds a new analysis plugin called analysis_nori that exposes the Lucene Korean analysis module - https://github.com/elastic/elasticsearch/pull/30397
Adding a char_group tokenizer - https://github.com/elastic/elasticsearch/pull/24186
Add Kerberos authentication support - https://github.com/elastic/elasticsearch/pull/32263
ML. Detectors now support custom rules that enable the user to improve machine learning results by providing some domain-specific knowledge in the form of rule. - https://github.com/elastic/elasticsearch/issues/31110
ML. Reverse engineer Grok patterns from categorization results - https://github.com/elastic/elasticsearch/pull/30125
Add support for field aliases - https://github.com/elastic/elasticsearch/pull/32172
Add an option to split keyword field on whitespace at query time - https://github.com/elastic/elasticsearch/pull/30691
The new _ignored field enables you to know which fields got ignored at index time because of the ignore_malformed option - https://github.com/elastic/elasticsearch/pull/29658
Add more contexts to painless execute api https://github.com/elastic/elasticsearch/pull/30511
Index phrases https://github.com/elastic/elasticsearch/pull/30450
Add a format option to docvalue_fields. - https://github.com/elastic/elasticsearch/pull/29639

Enhancements

Add exclusion option to keep_types token filter - https://github.com/elastic/elasticsearch/pull/32012
Added lenient flag for synonym token filter - https://github.com/elastic/elasticsearch/pull/31484
Consistent encoder names - https://github.com/elastic/elasticsearch/pull/29492
Add support for ignore_unmapped to geo sort - https://github.com/elastic/elasticsearch/pull/31153
Add Index UUID to /_stats Response - https://github.com/elastic/elasticsearch/pull/31871
INGEST: Extend KV Processor - https://github.com/elastic/elasticsearch/pull/32232/files
date_index_name processor template resolution. https://github.com/elastic/elasticsearch/pull/31841
Introduction of a bytes processor - https://github.com/elastic/elasticsearch/pull/31733
Extend allowed characters for grok field names - https://github.com/elastic/elasticsearch/pull/31653
Ingest: Add ignore_missing option to RemoveProc - https://github.com/elastic/elasticsearch/pull/31693
Add region ISO code to GeoIP Ingest plugin - https://github.com/elastic/elasticsearch/pull/31669
Add ingest-attachment support for per document indexed_chars limit - https://github.com/elastic/elasticsearch/pull/31352
Return statistics about forecasts as part of the job stats and usage API - https://github.com/elastic/elasticsearch/pull/31647
[ML] Add description to ML filters - https://github.com/elastic/elasticsearch/pull/31330
Remove RestGetAllMappingsAction https://github.com/elastic/elasticsearch/pull/31129
_cluster/state should always return cluster_uuid - https://github.com/elastic/elasticsearch/pull/30143
Rename ranking evaluation quality_level to metric_score - https://github.com/elastic/elasticsearch/pull/32168
Rename ranking evaluation response unknown_docs section - https://github.com/elastic/elasticsearch/pull/32166
Add Expected Reciprocal Rank metric - https://github.com/elastic/elasticsearch/pull/31891
Add details section for dcg ranking metric - https://github.com/elastic/elasticsearch/pull/31177
Add second level of field collapsing - https://github.com/elastic/elasticsearch/pull/31808
Added max_expansion param to span_multi - https://github.com/elastic/elasticsearch/pull/30913
Store username on watch execution - https://github.com/elastic/elasticsearch/pull/31873

Bug fixes

Ensure that ip_range aggregations always return bucket keys. - https://github.com/elastic/elasticsearch/pull/30701
Add support for indexed shape routing in geo_shape query - https://github.com/elastic/elasticsearch/pull/30760
Add support for is_write_index in put-alias body parsing - https://github.com/elastic/elasticsearch/pull/31674
Make sure that field collapsing supports field aliases - https://github.com/elastic/elasticsearch/pull/32648
Get Mapping API to honour allow_no_indices and ignore_unavailable - https://github.com/elastic/elasticsearch/pull/31507
Fix _cluster/state to always return cluster_uuid - https://github.com/elastic/elasticsearch/pull/30656
Reject forcemerge requests with a body - https://github.com/elastic/elasticsearch/pull/30792

New API’s

There a quite a few new API’s that need to be mapped:

xpack.license.get_basic_status.json
xpack.license.post_start_basic.json
xpack.ml.delete_calendar.json
xpack.ml.delete_calendar_event.json
xpack.ml.delete_calendar_job.json
xpack.ml.get_calendar_events.json
xpack.ml.get_calendars.json
xpack.ml.info.json
xpack.ml.post_calendar_events.json
xpack.ml.put_calendar.json
xpack.ml.put_calendar_job.json
xpack.ml.get_calendar_job.json
xpack.rollup.delete_job.json
xpack.rollup.get_jobs.json
xpack.rollup.get_rollup_caps.json (https://github.com/elastic/elasticsearch/pull/30401)
xpack.rollup.put_job.json
xpack.rollup.rollup_search.json
xpack.rollup.start_job.json
xpack.rollup.stop_job.json
xpack.sql.clear_cursor.json (https://github.com/elastic/elasticsearch-net/pull/3414)
xpack.sql.query.json (https://github.com/elastic/elasticsearch-net/pull/3414)
xpack.sql.translate.json (https://github.com/elastic/elasticsearch-net/pull/3414)
xpack.ssl.certificates.json
scripts_painless_execute.json
xpack.ml.update_filter.json
xpack.rollup.get_rollup_index_caps.json
xpack.security.delete_privileges.json
xpack.security.get_privileges.json
xpack.security.has_privileges.json
xpack.security.put_privilege.json
xpack.security.put_privileges.json

Tests.Throughput

An unscientific throughput application to measure our different IConnection implementations on how many messages we can pump. For now we only plan to measure _bulk and _search and we’ll use the machine learning data set MachineLearningCluster already pulls down.

Will warrant its own [meta] ticket which will be linked here later.

Netcoreapp2.1

or 2.2 whichever comes first ðŸ˜¸.

Explicitly ship this TFM and get some numbers how the managed handler performs on linux/windows through Tests.Throughput vs the already existing implementations/
Move our dotnet core IHttpConnection over to HttpClientFactory. We currently do much of the same but if we don’t have to maintain our own pooling of HttpClient instances that would eleviate some complexity in our codebase.