Search freeze on first search

Description

When ever we open the page first time and click on search box, it freezes for almost 10 seconds.

Expected behavior

Should be instant like in other themes OR show what it is doing.

Actual behavior

Freezing the whole window

Steps to reproduce the bug

1. I’ve around 20 MD files with 5000 lines of text.
2. Open main page and click search box
3. Wait and enjoy the slow speed.

In this screencast, i pressed button instantly but results were way too late. https://i.imgur.com/KZU3G66.gifv

I changed theme to readthedocs, and tried as well which does not freeze. Let me know what i am missing.

Package versions

• Python: 3.7
• MkDocs: 1.0.4
• Material: 3.0.4

Project configuration

site_name: ................
site_description: .......... Documentation
# site_url:
site_author: '..............'
site_url: 'https://........../docs'

# Copyright
copyright: 'Copyright © 2010 - 2018'

nav:
    - Getting Started:
      - 'Setup' : 'index.md'
      - 'Creating' : 'creating.md'
    - Customization:
      - 'Selecting Columns' : 'selecting-columns.md'
      - 'Column Options' : 'column-options.md'
      - 'Options' : 'options.md'
    - Features:
      - 'Master Detail' : 'master-detail.md'
      - 'Sub' : 'sub.md'
      - 'Exporting Data' : 'exporting-data.md'
      - 'Importing Data' : 'importing-data.md'
      - 'Conditional Formatting' : 'conditional-formatting.md'
      - 'Grouping' : 'grouping.md'
      - 'Grouping Headers' : 'grouping-headers.md'
      - 'Search' : 'search.md'
      - 'Events' : 'events.md'
      - 'Validation' : 'validation.md'
      - 'Upload Files' : 'upload-files.md'
      - 'Summary' : 'summary.md'
      - 'Autocomplete' : 'autocomplete.md'
    - Layout & UI:
      - 'Responsive Design' : 'responsive-design.md'
      - 'Localization' : 'localization.md'
      - 'Themes' : 'themes.md'
    - Databases:
      - 'Connecting with SQL Server' : 'connecting-with-sql-server.md'
      - 'Connecting with Postgres' : 'connecting-with-postgres.md'
      - 'Connecting with Oracle' : 'connecting-with-oracle.md'
      - 'Debug Mode' : 'debug-mode.md'
    - FAQs:
      - 'General' : 'faq-general.md'
      - 'Appearance' : 'faq-appearance.md'
      - 'Column Appearance' : 'faq-column-appearance.md'
      - 'Column Settings' : 'faq-column-settings.md'
      - 'Dialog Customization' : 'faq-dialog-customization.md'
      - 'Javascript API' : 'faq-javascript-api.md'
      - 'Databases' : 'faq-databases.md'
      - 'Master Detail' : 'faq-master-detail.md'
      - 'Export' : 'faq-export.md'
      - 'Misc' : 'faq-misc.md'
theme:
    logo: 'https://.............png'
    name: material
    favicon: 'css/favicon.gif'

# Customization
extra:
  feature:  
    tabs: true
  search:
  #   tokenizer: '[\s\-\.]+'
    language: 'en'
    # prebuild_index: true

site_dir: ../docs
docs_dir: md


extra_css:
    - extra/extra.css
extra_javascript:
    - extra/extra.js

System information

• OS: Win10
• Browser: Chrome Version 70.0.3538.67 (Official Build) (64-bit)

[squidfunk]

This problem has been known for some time and it is, of course, annoying. The delay is actually caused by three factors: (1) the download of the search_index.json, (2) pre-processing the index data and (3) the actual indexing. Let me elaborate on how we could cut time and what it would cost, annotating the separate steps with the times I measured for the official docs:

Process

1. Download (74ms)

Downloading the search_index.json is only done when the search is first focused. The download is network time, so it depends on two main factors, namely the connection quality and the size of the index. The index contains all documents two times: the whole document, as well as all separate subsections of a document. I don’t know why the authors choosed to do that, probably to allow whole-page or sub-section indexing.

Material takes advantage of the presence of both, as it groups sub-section results by page, which is a major UX improvement over the default MkDocs search, especially if you have a lot of pages with the same sub-section names (e.g. try to search the term installation in the official docs). Therefore, size may be a critical factor that impacts search performance. It can be improved with appropriate caching headers and gzip encoding.

2. Preprocessing (5ms)

The pre-processing of the index is a Material-specific feature. The Markdown is filtered for better readability within the search results and sections are annotated with their respective page, see:

https://github.com/squidfunk/mkdocs-material/blob/fa8d490b16155003386224cc82e71aafb471c828/src/assets/javascripts/components/Material/Search/Result.jsx#L137-L164

This seems to take only very little time as it is contained in the block prior to the anonymous function passed to lunr.js.

3. Indexing (150ms)

This takes the most time, as lunr.js has to tokenize the words and do a lot of string processing operations. After lunr.js is finished, the search is operational.

Mitigation

The problem is actually two problems: (1) the building of the index takes too long and (2) it freezes the UI. Both could be mitigated, but it comes at a cost:

1. Improving on indexing time

The indexing time could be improved by pre-building the index which is already possible with lunr.js. However, as Material does pre-processing for an optimal delivery of search results, this pre-processing must also be carried out during the prebuild. It’s all JavaScript, so it is easy to be done with Node.js.

However, this would mean that Material would need to supply custom scripts which must be executed during the actual build of the documentation project which would in turn add a dependency on a functioning Node.js environment and some additional setup. This is something that I really don’t want to do, because it would make things more complicated than just spinning up MkDocs with the packaged Material theme for every user.

Frankly, I don’t really know how we could come up with a way that doesn’t complicated the setup for delivering a prebuilt index. If somebody has an idea, feel free to discuss.

2. Moving search into a separate thread

We could (and should!) leverage web workers to make the search index processing asynchronous. The UI wouldn’t freeze, so the UX would be much better. However, this would probably increase the index building time (as it’s done with sub-priority) which means that we wouldn’t gain much here, either. Neverheless, if somebody wants to go ahead on web worker support, let’s discuss and PR.

Conclusion

As said, it’s a know problem, but I haven’t come up with a one-fits-all solution yet. Note that I elaborated on the way search is carried out for future reference in the form of similar questions.

[Kludex]

Thank you! 😃