Cspell performances with large (7k) words list in cspell configuration

See original GitHub issue

Info

Kind of Issue

runtime - command-line tools
building / compiling
security
change in behavior
crash / error

Which Tool or library

cspell – the command-line spelling tool
cspell-tools – used for building dictionary files
cspell-lib – library that does the actual spell checking.
cspell-trie – tool for working with trie files.

Which Version

Version: 5.16.0

Issue with supporting library?

No
cspell-glob – library for matching glob patterns
cspell-io – thin file i/o library
cspell-trie-lib - trie lib
cspell-trie2-lib - trie lib alternate format

OS:

Macos
Linux
Windows
Other

version:

python:3.9.7-alpine3.13

Bug Description

Describe the bug

I have a repo where there are 1500+ files to check for spelling mistakes Before I added a lot of words in .cspell.json, performances were almost acceptable But now I added a big .cspell.json, it takes 250 seconds Notes:

the 1500 files are sent as cspell arguments: cspell file1 file2 file3 ...
my dockerfile already contains ENV NODE_OPTIONS="--max-old-space-size=8192"

Is it an expected performances or are there ways to improve them ?

Thanks for your tool and your answer 😃

Issue Analytics

State:
Created 2 years ago
Comments:9 (2 by maintainers)

Top GitHub Comments

1reaction

Jason3Scommented, Jan 31, 2022

I’m going to close this for now, since it is now 4-5x faster than 5.17

1reaction

Jason3Scommented, Jan 30, 2022

@nvuillam,

I was able to speed it up a bit by caching some of the internal word lists. It is still 2x slower than using a custom dictionary.

You can try it out: npx cspell@next.

I’ll release 5.18.0 tomorrow.

Top Results From Across the Web

Configuration | CSpell

CSpell's behavior can be controlled through a config file. ... flagWords - a list of words to be always considered incorrect; ignoreWords -...

Configuration Settings | Spell Checker - Street Side Software

Determines if words must match case and accent rules. cSpell.customDictionaries, resource, Custom Dictionaries. cSpell.dictionaries, resource, Optional list of ...

cspell - npm

Customizable -- supports custom dictionaries and word lists. Continuous Integration Support -- Can easily be added as a linter to Travis-CI.

Automated docstring and comments spell check - Stack Overflow

Yeah, there could a very-very long list of words to ignore ( spelling-ignore-words setting) which can be eventually difficult to maintain. I'll give...

streetsidesoftware/cspell: A Spell Checker for Code! - GitHub

... schema for cspell configuration files. cspell-tools -- tool used to compile dictionaries. cspell-trie-lib -- trie data structure used to store words.