Message files splitting

It seems that one unsolved problem in JavaScript internationalization is automatic splitting of message files, similar to code splitting.

I would like to tackle this problem in v3 and figure out, what API changes we need to make to get it done.

Related issues: https://github.com/lingui/js-lingui/issues/474 https://github.com/lingui/js-lingui/issues/362

---

Code splitting creates main bundle with loader and then smaller chunks, which can be loaded separately. We know, what modules are included in each chunk (webpack stats data) and we know, what messages are included in each module (origin of message in lingui catalogs). Therefore we can determine what messages are included in each chunk and generate message files for each chunk.

As with code splitting, even locales need to have main bundle, which will include locale specific data like plural rules, date/number formats, etc.

The only question is, how to load chunks of message files.

Webpack handles loading of code chunks automagically (== I have no idea how it works). When we type import "module" and the module isn’t loaded yet, webpack automatically loads the correct chunk.

We can’t do that with locales. Usually, we would write import "locale/en.json", but that would load whole catalog. In fact, we need just import "locale/en.acf422.json", but that filename is generated automatically. We probably need to hook into webpack require and when chunk xyz is loaded, then locale locale/en.xyz.json must be loaded as well.

Finally, the last piece in puzzle - once we have locale chunks loaded automatically, we need to load them into Lingui object.

---

Notes

While trying to figure out how to load message catalog automatically with code chunk, I discovered next-client-pages-loader from Next.js, which injects some extra code to generated chunk.

We need to either override default webpack require or call another require before/after chunk is loaded, so adding extra code to generted bundle might be related.

---

Checklist:

• Generate locale files for each chunk
• Figure out, how to hook into webpack’s require and load locale chunk along with code chunks
• Figure out how to feed loaded locale chunks into Lingui

[JSteunou]

nooo stale bot please keep it up keep it up 😃

[tricoder42]

@semoal I’ve just created a milestone and added two new possible features.

This issue isn’t trivial because it depends on the framework. I would start with Next.js because it’s closer to me. Just clarify — we’re talking about splitting the compiled catalog, not the source one. We can already split the source catalog using catalogs attribute and, to be hones, even one large source catalog isn’t an issue. The problem is when you send such large catalog to the client, which potentially might explode the bundle size.

It would be great, if we could split the catalog based on generated chunks, but I have no idea how to hook into Webpack. @ScriptedAlchemy mentioned module federation yesterday.