Allow settings in mkdocs.yml to alter search behaviour

I checked that…

• … the documentation does not mention anything about my idea
• … to my best knowledge, my idea wouldn’t break something for other users
• … there are no open or closed issues that are related to my idea

Description

Extend the search capabilities using the built in clauses in lunr to include strict or fuzzy matches.

Disclaimer I’m not that good at JavaScript, so definitions and methods may not be 100% correct, but it’s my interpretation of the source code.

Use Cases

Depending on the knowledge level of a user of what to search for, a strict term presence may increase matching for experienced users when using multiple terms, or a fuzzy match when the target user group may be less knowledgeable. Taking stemming into account, a strict term presence can assist in separating hyphened words.

Mockups

The search capabilities should be defined in the mkdocs.yml file.

search:
    method: (undefined)/default/strict/fuzzy

(undefined)/default: The default/current search behavior is used, wildcard: lunr.Query.wildcard.TRAILING strict: The search query should use the option presence: lunr.Query.presence.REQUIRED fuzzy: The search query should use the option editDistance: 1

If no method is set, the search method should use default.

---

The rendered HTML should include the search method in the same manner as the tokenizer and languages are defined. This could be omitted if undefined or set to default.

<meta name="lang:search.method" content="default">
<meta name="lang:search.method" content="strict">
<meta name="lang:search.method" content="fuzzy">

---

The search method is retrieved in the Results.jsx file, perhaps in the constructor where the tokenizer and languages are defined, line 153.

/* Load search method */
const method = translate("search.method")
if (method === "strict")
	/* relevant code to set the options for strict. */
elseif (method === "fuzzy")
	/* relevant code to set the options for fuzzy. */
else
      /* relevant code to set the default search options  */

The search query, I guess it’s line 288, should be altered from using a static option to a dynamic option based on the set search method.

...
query.term(term, { wildcard: lunr.Query.wildcard.TRAILING })
...

Backwards compatibility

If no method is set, the search method should be default, thus not requiring users of Material for MkDocs to alter their mkdocs.yml-file

Performance implications

Unsure if there will be any major implications of performance

References

https://lunrjs.com/guides/searching.html https://lunrjs.com/docs/lunr.Query.html https://lunrjs.com/docs/query.js.html#line76

[squidfunk]

I don’t quite share waylan’s view here. I think that users should get the optimal experience right out of the box, whereas optimal is something the content author should decide. I would assume that only a small fraction of the people using Google search know they can exclude a term with a - prefix.

For this reason I think this is a good addition and will reopen this issue for further consideration. I just wanted to run it by waylan in order to check whether it could be considered for upstream integration, so all themes could benefit from it. If I remember correctly, tokenization was also added as a configuration option long after Material supported it.

The downside of this is that search configuration is split across plugins and theme or extra. I’ll try and come up with a good configuration proposal for v5.

[squidfunk]

I’ve prototyped the new search locally (not yet committed, needs cleanup) and I think I came up with a reasonably customizable approach. I think waylan is right - we should just use the search function, giving users the possibility to use all features that lunr supports.

However, I also want to keep the current trailing wildcard behavior as it has proven to be a very good default for most of the cases. To see it in action, compare Material’s search with the one on mkdocs.org which doesn’t add implicit trailing wildcards.

For this reason, I think the best approach is to take the input string from the search query field and apply the following transformation:

query.trim().replace(/\s+|$/g, "* ")

This will lead to the following transforms:

foo bar       => foo* bar*
foo -bar      => foo* -bar*
foo +bar      => foo* +bar*

Of course this needs some further validation, for example whether and how we can handle fuzzy queries with that. To change the default behavior, we could add a function to the configuration which the Material application receives in base.html, so the author can override the default query transform behavior.

@martinbira any thoughts?