Can't understand "word_delimeter" token filter

elasticsearch-dsl==6.2.1 elasticsearch==6.3.1

I have analyzer:

test_analyzer = analyzer(
    'test_analyzer',
    tokenizer=tokenizer('trigram', 'edge_ngram', min_gram=3, max_gram=10),
    filter=['lowercase', 'word_delimiter']
)

This analyzer used in Document:

class TestIndex(Document):
    name = Text(analyzer=test_analyzer)
    id = Integer()

    class Index:
        name = 'test-index'

Objects in my Index have next pattern in name:

word1:word2:word3:word4

(there a lot of “:” in name)

As I understood ES docs, with this analyzer I can search my objects by sub-words (like word2 from my example), but really search works only with almost-full-name in query.

My Search request is:

search = Search(
    index='test-index'
).query(
    "multi_match",
    query="word2",
    fields=['name'],
    fuzziness='AUTO'
)

(returns nothing)

[honzakral]

The problem is with your tokenizer which produces just edge ngrams of length 3-10 from the original input (word1:word2:word3), in your case:

wor
word
word1
word1:
word1:w
word1:wo
word1:wor
word1:word

which is not particularly useful I believe. I would recommend you play around with the _analyze API (0) to find an analyzer that does what you want, in this case I believe you want the simple_pattern_split tokenizer (1) instead:

es.indices.analyze(body={
    'text': 'word1:word2:word3',
    'tokenizer': tokenizer('split_words', 'simple_pattern_split', pattern=':').get_definition()
})

Hope this helps!

0 - https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html 1 - https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html

[honzakral]

can you just print out the result instead of the assert? My guess would be that the code is accidentally run twice so the document is duplicated. Or just print out the hits instead of just a count().