Mark Text performance improvement

Description

At the moment Mark Text using a simple String to represent the document content. When it comes to larger documents, this is very inefficient and result in high CPU and RAM usage because we create many copies from the String.

A solution would be to use a (immutable) data structure with support for snapshots (undo/redo) and better memory management. I don’t want to go into the detail, but the explanation from Juan Pedro Bolivar Puente of his talk “Postmodern immutable data structures” or Atom’s/vscode’s blog post, does indeed. There are already libraries, which we can use like Superstring from the Atom text editor.

Problems

• How do we interact with CodeMirror and the virtual HTML DOM stuff. Should we fork/implement this by our own and consider this problem?
• Can we “just” change your API to use a different data structure?

Note to self: Atom has written a blog post about integrating Superstring with a virtual DOM.

Existing implementations

• Superstring - Native library at the core of Atom’s text editor.
• Postmodern immutable and persistent data structures for C++
  • CppCon’17 Talk: Postmodern Immutable Data Structures Talk
• immutable.js - Immutable persistent data collections for Javascript
• CodeMirror’s text buffer

References

• Atom’s new concurrency-friendly buffer implementation
• Text Buffer Reimplementation

[VegaDeftwing]

Using Marktext 0.14.0-3 – marktext-bin in the AUR on a very high end system

Wanted to drop by and note that this is actually affecting real world use cases. I’m working on a massive markdown document over at https://github.com/VegaDeftwing/opinionatedeng/blob/master/opinionatedeng.md and hoboy does marktext get laggy, but then again so does everything else. Even just a raw text editor, like VSCode, with syntax highlighting for the markdown just gets bogged down.

I haven’t dug though the code at all so I’m not sure if these are done already or the feasibility/practicality of any of these recommendations but:

• If images are rendered at all times, maybe only rendering them when on screen would help
• Adding the option to disable some of the more advanced options (Vega Chart, Flow Chart, Sequence Diagrams) may help
• As noted about history, anything with the clipboard is awful, and from the original issue description above, it does sound like this just comes down to the data structure used. You may be able to abuse some things about the data though. For example, it’s almost certain that any document large enough to notice a performance impact will be organized by headings, maybe you could split at each header? I’m not sure this is a logical route to take, but I think it’s worth considering
• Similarly, the data is fundamentally raw text data and working with the MD file directly may be the obvious route, but there’s no reason a cache where the file is converted to something more processing friendly couldn’t be kept: say in JSON somewhere?

I know that many of these suggestions hit the same issue as described above- a significant refactor is necessary no matter what, even if just switching to a better data structure as originally suggested.

Thanks for all the great work on this project 😃

EDIT: just checked the roadmap and saw that it’s at the top and already well planned out. Thanks again to everyone working so hard on this!

[bigshans]

Source Code Mode is much slower than WYSIWYG mode.