[Discuss]Problems about unstable numeric module ID.

Do you want to request a feature or report a bug? Feature, or discuss first.

What is the current behavior?

Modules are now packed by unstable numeric module ID. There are some bad parts of this behavior:

1. Unfriendly to diff.

Supposing there is an entry which some deps like following:

import a from "./lib/a";
import b from "./lib/b";
import c from "./lib/c";
import d from "./lib/d";

Bundle it first. Then remove one of deps, for example, ./lib/a:

-import a from "./lib/a";
import b from "./lib/b";
import c from "./lib/c";
import d from "./lib/d";

Bundle it again. Check bundle’s diff:

As you see, all define/require code (module ID, exactly) about deps after ./lib/a were changed.

This may give rise to lots of unecessary mutated in some projects’s bundle which has huge dependency graph. It’s not friendly to bundle diff/update. (Some Apps and 3rd. tools will update bundle by diff algorithm to reduce the consumption from un-splitting bundle size.)

2. Can’ t run muti-bundle(RN App) at one context.

In our team, there are many native Apps which have been integrated React Native paritially, and each native App may include mutiple RN Apps (with mutiple entries).

We are finding a way that can run mutiple RN Apps in a common JS context so that we don’t need to recreate bridge/context, run bundle and define method/modules over and over, even can share data and modules with each other.

Think about the style of Node.js server application, we don’t need to redefine modules like react, react-dom, react-router blah-blah for each route when it called by requests.

But at least now, It can’t be implemented because of numeric module ID. Bundles will override other’s modules which always start with 0, 1, 2 but not an unique ID.

3. Unfriendly to bundle-splitting (module reference).

We are also trying to split bundle for decrease update download size. You can see an earlier incomplete PR facebook/react-native#10804 I’ve sent.

We splited most of library modules, like react, react-native and our common utils into a base bundle, only pack business code into business bundle, then maintaining a manifest.json file to handle module call from business bundle to base bundle (Like webpack’s DLL way).

But you can guess that numeric module ID also make it unstable. If we add a module, or reorder the imports to any of deps in base bundle, manifest will get a huge change. We have to repack all business bundles. This means that we can’t make a stable module reference by current packager.

If the current behavior is a bug, please provide the steps to reproduce and a minimal repository on GitHub that we can yarn install and yarn test. Not a bug.

What is the expected behavior?

Could metro-bundler support string module ID?

Support string module ID so that we can get more stable bundle, and more friendly to diff, make stable reference and work on bundle-splitting.

We know that each module has an unique path relative to project’s root path. Maybe we can create a hash by the path in moduleIdFactory, to get an unique stable ID.

And if we want to create a bundle with isolate IDs (run different projects bundle, or upgrade module partially), we also can add a prefix to the moduleIdFactory when pack the bundle. It’s absolutely more extensible than numeric.

I also noticed about the indexed-ram-bundle in unbundle way. It has relied on numeric ID strongly AFAIK. Could metro-bundler make compatible with this two ways? Or through option to enable stable string module ID?

looking forward to your opinions @cpojer @jeanlauliac @amasad @davidaurelio. And willing to send PR if needed!

[mc-zone]

Thanks for your reply! @davidaurelio

Yes, I probably knew we should retain numeric module ID for Random Access Bundle because of it’s special index table. Now your reply also help me realize more about it.

How about when using normal bundle, except unbundle, provide an option to let users choose, or define their own moduleIdFactory?

I can realize about bundle size increase that your’ve mentioned. But I think it maybe not so much.

In fact, our team are using a local fork of react-native based on 0.35, changed the createModuleIdFactory to use hash module ID. code like:

function createModuleIdFactory({projectRoots, prefix}) {
  const fileToIdMap = Object.create(null);
  const usedIds = {};
  return ({path: modulePath, name}) => {
    var relativePath = getPathRelativeToRoot(projectRoots, modulePath);
    if (!(modulePath in fileToIdMap)) {
      fileToIdMap[modulePath] = getModuleHashedPathId(relativePath, usedIds);
    }
    return fileToIdMap[modulePath];
  };
}
function getModuleHashedPathId(path, usedIds){
  var len = 4;
  var hash = crypto.createHash("md5");
  hash.update(path);
  var id = hash.digest("base64");
  while(usedIds[id.substr(0, len)]){
    len++;
  }
  id = id.substr(0, len);
  usedIds[id] = path;
  return id;
}

Part of the output bundle like this:

Now I’ve make a compare at one of our RN apps, which total 539 modules in production. Here is the size change betwteen numeric module ID and hash module ID (both with uglify, dev=false):

838KB - 847KB. only 10KB increased.

So in contrast, I prefer to use hash module ID for normal text bundle, it give us ability to split bundle by stable module ID.

What you can do as of today is to implement the postProcessModulesForBuck hook to order modules by path.

I haven’t tried postProcessModulesForBuck yet. I will have a look at it. But I think maybe reorder modules by path can’t keep stable when deps was added/removed, their are still numbers related.

If I have any mistake please correct me. Thanks again!

[rafeca]

Hey @mc-zone, @lishoulong : We recently merged https://github.com/facebook/metro/commit/f347e4ff470ca858cdb8af5a80d409cc5061e9ee, which allows to customize the createModuleIdFactory() function, so if you use v0.24.1 you’re going to be able to use your custom logic for generating the module ids.