consider migrating from Immutable.js "Records" to plain objects

Do you want to request a feature or report a bug?

Discussion.

What’s the current behavior?

When Slate was first created, Immutable.js was the best and most popular way to handle immutability in React. However, it has many downsides for our use case:

• CON: It requires a fromJS step to build the collections/records. Since the objects aren’t the native JavaScript data types you get from JSON, we have to have an interim step that instantiates the Immutable.js data model. This can be costly for large documents, and there is no way around it. This is especially problematic in server-side environments where serializing to/from JSON blocks Node’s single thread.

• CON: Reading values is more expensive. Immutable.js is optimized for non-crazy-slow writes, at the expense of read operations being much slower than native JavaScript. Since Slate’s model is a tree, with many node lists, this ends up having a significant impact on many of the common operations that take place on a document. (See this question for more information.)

• CON: It introduces a fairly steep learning curve. People getting started with Slate often get tripped up by not knowing how Immutable.js works, and its documentation isn’t easy to understand. This results in lots of “wheel reinvention” because people don’t even realize that helper methods are available to them.

• CON: It makes debugging harder. In addition to a learning curve, debugging Immutable.js objects adds extra challenges. There’s a browser extension that helps, but that’s not good enough in lots of places, and you end up having to us toJS to print the objects out which is very tedious.

• CON: It increases bundle size. The first increase in size comes from just including the immutable package in the bundle at all, which adds ~50kb. But there is also a more insidious bundle size increase because the class-based Record API encourages monolithic objects that can’t be easy tree-shaken to eliminate unused code. Whereas using a utility-function-based API, similar to Lodash, would not have this issue.

However, it does have some benefits:

• PRO: Custom records allow for methods/getters. This has been the primary way that people read values from Slate objects. Because Slate (and rich text in general) deals with some complex data structures, having common things packaged up as methods allows us to reduce a lot of boilerplate and reinventing the wheel for common use cases.

• PRO: It offers built-in concepts like OrderedSet. These allow for more expressive code in core than otherwise, because we’d need to reinvent the wheel a bit to account for JavaScript not having some of these concepts. Although truth be told this is probably a fairly minimal gain.

---

Since Slate was first created, immer has come on the scene (thanks to @klis87 for kickstarting discussion of it in #2190) which offers a way to use native JavaScript data structures while maintaining immutability. This is really interesting, because it could potentially have big performance and simplicity benefits for us.

All of the CON’s above would go away. But we’d also lose the PRO’s. That’s what I’m most concerned about, and what I’d like to discuss in this issue… to see what a Slate without Immutable.js might look like, and how we could mitigate losing some of its benefits.

Without the ability to use classes and prototypes for our data models, we’d need to switch to using a more functional approach—exporting helpers in a namespace, like we currently already do for PathUtils. One question is whether this will be painful…

Looking at our rich-text example, we’d need to change how we do things in several places.

https://github.com/ianstormtaylor/slate/blob/8eb8e26958ef6c9337e838bfd46fc2cc55d7fe51/examples/rich-text/index.js#L54

We no longer have getters on our models, so value.activeMarks doesn’t work. Instead, we’d need to change this to:

return Value.getActiveMarks(value).some(mark => mark.type == type)

Similarly, there’s no value.blocks any more:

https://github.com/ianstormtaylor/slate/blob/8eb8e26958ef6c9337e838bfd46fc2cc55d7fe51/examples/rich-text/index.js#L66

So we’d need:

return Value.getClosestBlocks(value).some(node => node.type == type)

This is actually nice because we’re no longer using potentially expensive getters to handle common use cases—calling functions is more clear.

But we also can’t use helper methods like document.getParent.

https://github.com/ianstormtaylor/slate/blob/8eb8e26958ef6c9337e838bfd46fc2cc55d7fe51/examples/rich-text/index.js#L148

Instead we’d need to use:

const blocks = Value.getClosestBlocks(value)
const parent = Node.getParent(value.document, blocks[0].key)

Similarly, we can’t do:

https://github.com/ianstormtaylor/slate/blob/8eb8e26958ef6c9337e838bfd46fc2cc55d7fe51/examples/rich-text/index.js#L293-L295

And would have to instead do:

const blocks = Value.getBlocks(value)
const isType = blocks.some(block => {
  return !!Node.getClosest(document, block.key, parent => parent.type == type)
})

But we also get to remove some expensive code, since we don’t need to deserialize anymore:

https://github.com/ianstormtaylor/slate/blob/8eb8e26958ef6c9337e838bfd46fc2cc55d7fe51/examples/rich-text/index.js#L41-L43

That’s all for the rich text example, but it would definitely be a big change.

---

I’m curious to get other folks’s input. Are there other PROS or CONS to Immutable.js that I haven’t listed above? Are there other ways of solving this that I haven’t considered? Any thoughts! Thank you!

[renchap]

If you go with those big changes, might I suggest starting to use Typescript for Slate’s core? It is not hard to incrementally use typescript nowadays as its only a Babel plugin (I did a coffee => js => ts migration for my app incrementally, I dont regret it at all and both can really cohabit).

[ianstormtaylor]

@grsmvg: But now that Immutable comes in a package deal with Slate, I find myself using quite some Immutable functions already, like value.document.nodes.get(-2) to get the second last node. This can also be achieved with (slightly more verbose) plain javascript or an imported utility library but it would definitely be quite a rewrite. I expect I’m not the only one who would need to rewrite quite some bit.

That’s true, there are definitely some methods in Immutable.js that are useful. But it is definitely something that most people never find. Whereas if we using plain JavaScript data structures people would end up using lodash, or whatever else they liked, for all of these types of things. And as browser’s further improve their standard library they could remove more and more of the imports.

@grsmvg: On one hand I’m wondering if this change is really needed on the short-term, since it will probably delay v1 by another few months, without really adding new functionality (rather removing it). But on the other hand, my fear of needing to rewrite quite a bit of code only supports making this change now, instead of in between v1 and v2, because then it might impact even more projects in production.

I understand where you’re coming from here, but there is no timeline for 1.0 right now, on purpose. Because we want to be able to potentially make changes like this that are breaking (and sometimes tough) but greatly improve the library in the long run. Which is why I’ve kept Slate in “beta” and not made any promises on when 1.0 will land.

@skogsmaskin: In my opinion we need to focus on stabilising what we have now. There has already been two heavy refactors lately. Those were needed and gave very clear benefits right away. This one I’m not so sure about (I agree with much what @grsmvg said). I see why we would want it though, but I think it would be better doing it in a v2 when things are more stable all over.

Besides I think such a re-write would take a lot of focus and energy away from bug-fixing, cross browser compatibility (as fore slate-react), and general UX concerns. And introduce a whole lot of new bugs in the re-write phase.

This is fair, and I’m okay with waiting a little bit so that we can get the current 0.43/0.20 to be more stable. But if we decide to do it (and I’m leaning towards it more and more), it’s not going to end up something we do in 2.0 because the whole point of being in “beta” is that we are able to make these changes now before the library settles and more and more people depend on its API’s.

@renchap: A point you did not mention: not using Immutable will also reduce the size of Slate’s dependencies. If you are not already using Immutable in your app, it adds 55kb (minified, not gzipped) of JS to your bundles. It is really best for “core” libraries like Slate to come with as small as possible overall size (dependencies included).

This is a great point! Not only by being a dependency, but the extra bloat it adds to defining records would also reduce some of the core Slate bundle size too.

@ericedem: Is this something we could just achieve with build in javascript classes (which support getters)? If so, could Immer support them?

This is something I leaned towards at first, when we were focused mainly on Immer as the main question. But as it evolved, I realized that most of the benefits in terms of performance and simplicity actually come not from Immer, but from being able to use plain JavaScript objects to avoid parse/serialization steps, and to avoid having two different representations for objects.

At this point I’m more interested in the plain data structures than Immer itself. If there were some (unknown) better library that Immer, I’d use it while keeping the plain data structures.

---

@ppoulard I’m interested in Slate being able to support CRDT, however I have yet to see a CRDT approach that I think feels viable for nested documents and for real-world use cases where documents don’t balloon to huge sizes. For that reason most of Slate’s collaborative support has been focused on making OT possible first, which it already is today with the current codebase. (And it still would be with plain data structures.)

Since CRDT and collaboration is pretty convoluted, and seems so far like something that many people want, but few people actually have the time to do, or don’t want to put in the effort, it’s not something that I’m going to be personally adding to core. And for that reason I don’t want it to hamper any decision we can make here.

If it turns out to absolutely be the best way, and it turns out also that we absolutely have to use non-plain data structures, we can cross that bridge in the future when someone puts in the work to add CRDT. (But it might be very far in the future.)

---

Thank you everyone for your responses! (And feel free to provide more if you think of other things too.) The more I think about this, the more it seems like something that would be very beneficial to Slate. The tougher part is that decoupling from Immutable.js would be harder in cases where its custom (helpful) methods are used. That’s the biggest blocker in terms of a refactor.

I’m okay with waiting a little bit on this now, to let the current version of slate and slate-react stabilize from the recent two refactors. That way we have a more sound set of versions that people can wait on if they can’t refactor out Immutable.js right away, so we don’t leave them stranded. But this isn’t something that I think we’ll be waiting for 2.0 or 3.0 to do in years—that’s why Slate is in beta.

Thank you!