Pre-parsed dictionaries

Sometimes parsing a dictionary is a major bottleneck as that can take multiple seconds:

If it takes that long in the best case scenario the app’s spell checking, or whatever feature built on top of nspell, is going to feel broken, in the worst case scenario (single-core machines or machines where all other cores are busy) the app may become pretty unresponsive and slow for long enough that the user will just kill it.

I think a potential way to address this problem would be pre-parsing dictionaries, essentially skipping building the internal data structures from the raw dictionary file and loading them directly.

Would that be possible? If so could you give me some pointers on how I should go about implementing that? I’m thinking perhaps a serialize and deserialize methods could be implemented for this purpose.

[wooorm]

It sounds pretty complicated indeed 😅

I’m closing this because the actual problem should in part be solved by the PR, making this issue less of a thing!

[fabiospampinato]

@wooorm Yeah I’m already running nspell in a web worker, otherwise everything would just hang. I was actually thinking about using multiple web workers, I’m not sure if this is feasible because I’m not too sure about how the whole thing works, but do you think for example chunking the dictionary in 4 parts, making a nspell instance for each of them, and then managing them all would essentially parallelize the entire thing? Or would that not work for some reason?