the search ignores the symbols

Hello,

I added flexsearch library to my application. This works very well in the most cases. However, I have problems when I try to perform searches that combine words and symbols like &, !, #, %. For example:

We have a collection with 4 items:

1. Course&Some
2. Course#Another
3. Course%Foo
4. Course!Bar

When I try searching the term ‘Course&’ (ignore single quotes), the library returns 4 results, ignoring the symbol &. I expect one result (the item number one). When I try searching the term ‘Course&Another’ (ignore single quotes), the library returns 3 results (items two, three and four). I expect zero results.

So, my questions are:

1. How should I proceed so that the library does not ignore the symbols when searching?
2. Is there a way to improve the behavior of the library with cases like the ones mentioned?

I am using this configuration:

var index = new FlexSearch({
        encode: "advanced",
        tokenize: "full",
        depth: 3,
        threshold: 0,
        async: false,
        worker: false,
        cache: false,
        doc: {
            id: "Id",
            field: [
                "Field1",
                "Field2"
            ];
        }
    });
index.add(collectionItems);

I perform the search in the following way:

var params = { field: ["Field1", "Field2"], query: text, bool: "or" }

index.search(params, function (results) {
	//More code    
}

Thanks in advance.

[jladbury]

@carlinmack Thanks for taking an interest! I delved a little deeper and found that this works fine for my purposes:

tokenize: function(str){
	return str.split(/\s+/g);
}

I think it will also suit the riginal poster @javierperini.

[carlinmack]

I reckon the tokenize function full takes in your input and sanitises it, I would define a custom encoding function