Japanese Search problem in 5.2.3
See original GitHub issueDescription
We cannot search in Japanese with 5.2.3
Expected behavior
We can search in Japanese.
Actual behavior
We cannot search in Japanese.
Steps to reproduce the bug
- With 5.2.2, it works well
- With 5.2.3, it has the problem
git clone https://github.com/SI-Aizu/documentation.git
cd ./documentation
# 5.2.3
git checkout e55dc69
docker-compose up
# Go to http://localhost:8888
# Search "メンバー", it fails.
# 5.2.2
git checkout 6e359cf
docker-compose up
# Go to http://localhost:8888
# Search "メンバー", it will be found.
I am using my Docker image which is managed at peaceiris/mkdocs-material-boilerplate.
Package versions
- Python: 3.8
- MkDocs: 1.1.2
- Material: 5.2.3
Project configuration
https://github.com/SI-Aizu/documentation/blob/6e359cf037eb939bde922b677257aecbb18390ef/mkdocs.yml
# Project information
site_name: 'SI-Aizu Documentation'
site_description: 'SI-Aizu Documentation'
site_author: 'SI-Aizu'
site_url: 'https://si-aizu.github.io/documentation'
# Copyright
copyright: '© 2019 SI-Aizu'
# Repository
repo_name: 'GitHub'
repo_url: 'https://github.com/SI-Aizu/documentation'
edit_uri: 'https://github.com/SI-Aizu/documentation/edit/master/docs'
# Configuration
docs_dir: 'docs'
theme:
name: 'material'
language: 'ja'
shortcuts:
help: 191 # ?
next: 78 # n
previous: 80 # p
search: 83 # s
palette:
primary: indigo
accent: orange
# features:
# - tabs
#font:
#text: 'Ubuntu'
#code: 'Ubuntu Mono'
#icon:
#logo: 'images/logo.svg'
#favicon: 'assets/images/favicon.ico'
# Extensions
markdown_extensions:
- admonition
- codehilite
- footnotes
- mdx_math:
enable_dollar_delimiter: true
- toc:
permalink: true
separator: '_'
plugins:
- search:
separator: '[\s\-\.]+'
lang:
- ja
# Customization
extra_javascript:
- 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.6/MathJax.js?config=TeX-MML-AM_CHTML'
extra_css:
- 'assets/css/custom.css'
extra:
# manifest: 'manifest.json'
social:
- icon: 'fontawesome/brands/github-alt'
link: 'https://github.com/SI-Aizu/documentation'
System information
- OS: macOS, Linux (Ubuntu 18.04)
- Browser: Google Chrome
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Japan's DO NOT SEARCH Iceberg [Part 3] (Censored ...
Check out Ekster Wallets through this affiliate link! https://shop.ekster.com/sakurastardust MY NEW PATREON: ...
Read more >The Challenges of Intelligent Japanese Searching
To build a truly sophisticated, "intelligent" Japanese search engine, various challenges must be overcome. Here are some of the major issues:.
Read more >554 5.2.3 MailPolicy violation Error, help? - Zoho Cares
This message was created automatically by mail delivery software. A message that you sent could not be delivered to one or more of...
Read more >Japan -- Media Environment Open; State Looms Large
Like most Japanese press, it writes that Japan should maintain sanctions on the DPRK until there is "progress" on the abduction issue, but...
Read more >Addressed Issues in GlobalProtect App 5.2 - Palo Alto Networks
GlobalProtect App 5.2.3 Addressed Issues ... was set to Japanese, the password field for disabling the app was not properly displayed.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Released as part of 5.3.2.
I took this opportunity to revisit the multilingual search functionality. I’ve found that we didn’t quite correctly initialize the pipeline, which is now fixed in e6e75363. This leads to much better type-ahead behavior in many languages. I tested all of them, which took me quite a significant amount of time.
In general, we have two cases:
Languages for which
lunr-languages
provides support, likede
,es
, etc. These languages are configured to use the language-specific pipeline. The stemmer is removed, as it cripples type-ahead experience.Languages for which we don’t have dedicated support. This is the majority. Most of the languages work really well with the default configuration when removing the stemmer, for example
pl
. Some languages, likekr
don’t, but removing the entire pipeline seems to work well for them.Which leads me to the issue you reported. The search terms you provided now works again:
However, there may be some combinations of Japanese characters that don’t work, as was shown from my testing. This, however, appears to be an issue with
lunr-languages
and not Material for MkDocs. If you wish to follow up on this problem, I suggest you open an issue on thelunr-languages
repository. Other than that, this issue is considered fixed.I’ll issue a release shortly.