Elasticsearch: Unable to assign different boost values for same-named fields

The following definition is valid for Elasticsearch 1., but invalid for 2.:

    search_fields = Page.search_fields + [
        index.SearchField('title', partial_match=True, boost=10),
        index.SearchField('intro'),
        index.SearchField('body'),
    ]

With Elasticsearch 2 it causes the following error:

Mapper for [title] conflicts with existing mapping in other types:\n[mapper [title] has different [analyzer], mapper [title] is used by multiple types. Set update_all_types to true to update [boost] across all types.

I think we need to mention it somehow in documentation (maybe in the “Upgrade considerations” section?).

Important! The same issue can be reproduced with wagtail.wagtailadmin.taggable.TagSearchable. This class is deprecated, but will be removed only in Wagtail 1.8. That’s why I marked this issue as “Release blocker”.

Refs:

• #2570
• Conflicts between fields in different types

(cc @kaedroho)

[kaedroho]

Resolving this issue isn’t trivial, and it only affects ES1, which has been EOL since Janurary. We have good support for ES2 and ES5 so I think we should leave this one. Those who are affected by this issue should upgrade Elasticsearch.

[kaedroho]

One thing that might make implementing the query-time boost simpler could be to split the _all field into four separate fields for different rank levels. Eeach field is indexed twice, once with the usual field name and again in the global field that has the closest rank value.

When querying “all” fields, we query these four fields together setting the rank on the query.

This is basically how PostgreSQL FTS does ranking so restricting Wagtail to four levels of ranking should make it easier to integrate with that.

For example, the mapping of a page could be as follows:

{
    "properties" {
        "rank_a": {"type": "string"},
        "rank_b": {"type": "string"},
        "rank_c": {"type": "string"},
        "rank_d": {"type": "string"},
        "title": {"type": "string", "copy_to": "rank_a"},
        "body": {"type": "string", "copy_to": "rank_b"},
        "unimportant_field": {"type": "string", "copy_to": "rank_d"},
    }
}

When querying all pages, we only query the rank_* fields as we are not aware of any specific fields for all pages:

{
    "multi_match" : {
        "query": "hello world", 
        "fields": ["rank_a^1.0", "rank_b^0.7", "rank_c^0.5", "rank_d^0.1"] 
    }
}

And when querying specific fields directly, we could look up the actual rank of the field manually:

{
    "multi_match" : {
        "query": "hello world", 
        "fields": ["title^1.0", "body^0.8"] 
    }
}

This solves the problem with clashing boosts, it removes the need for index-time boosting (which makes field-length norms more accurate) and it also simplifies the ranking in Wagtail enough to make it fit much better with Postgres FTS.