unable to import or require 0.7.1 under `-r esm`

All of the agoric-sdk code is still using -r esm (we use import and export as if we were writing modules, but the package.json do not declare them as such, and we rely upon -r esm to let us load module-ish code anyways).

I tried to import ses as usual, and got an error which I don’t know how to work around. The minimal test case looks like:

warner@turing:/tmp$ mkdir t
warner@turing:/tmp$ cd t
warner@turing:/tmp/t$ yarn add ses@0.7.1 esm
yarn add v1.22.0
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 3 new dependencies.
info Direct dependencies
├─ esm@3.2.25
└─ ses@0.7.1
info All dependencies
├─ @agoric/make-hardener@0.0.6
├─ esm@3.2.25
└─ ses@0.7.1
Done in 0.39s.
warner@turing:/tmp/t$ cat >t.js
import { lockdown } from 'ses';
warner@turing:/tmp/t$ node -r esm t.js
/tmp/t/node_modules/ses/src/lockdown-shim.js:1
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /tmp/t/node_modules/ses/src/lockdown-shim.js
require() of ES modules is not supported.
require() of /tmp/t/node_modules/ses/src/lockdown-shim.js from /tmp/t/t.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename lockdown-shim.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /tmp/t/node_modules/ses/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:13) {
  code: 'ERR_REQUIRE_ESM'
}

I know that -r esm is a crutch, and that it doesn’t play nicely with real modules: I compiled a list of rules in https://github.com/Agoric/agoric-sdk/issues/527#issuecomment-587246535 , and I think we’re running afoul of at least one of them here. But I’m not prepared to change all of agoric-sdk yet, it’s a big project. I was hoping the boundary between agoric-sdk and SES-shim would let me avoid doing that.

I think esm is rewriting my t.js into CommonJS format, into something like const { lockdown } = require('ses');. If I write that into t2.js, I get a slightly different error:

warner@turing:/tmp/t$ cat >t2.js
const { lockdown } = require('ses');
warner@turing:/tmp/t$ node t2.js
internal/modules/cjs/loader.js:1167
      throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);
      ^

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /tmp/t/node_modules/ses/dist/ses.cjs.js
require() of ES modules is not supported.
require() of /tmp/t/node_modules/ses/dist/ses.cjs.js from /tmp/t/t2.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename ses.cjs.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /tmp/t/node_modules/ses/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:13)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)
    at Module.require (internal/modules/cjs/loader.js:1040:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/t/t2.js:1:22)
    at Module._compile (internal/modules/cjs/loader.js:1151:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14) {
  code: 'ERR_REQUIRE_ESM'
}

Note that t.js failed as it tried to require ses/src/lockdown-shim.js (which is listed in ses 's package.json as both main: and exports.import), while t2.js failed as it tried to require dist/ses.cjs.js (which is listed as exports.require).

If I run node -r esm t2.js, I get the same error as node -r esm t.js: unable to require src/lockdown-shim.js.

Any ideas?

[kriskowal]

We’ve addressed this issue with a cocktail of adjustments to how leaves on the dependency graph provide hybrid support for CJS, RESM, and ESM.

rollup config:

1. generate a dist .cjs file. The extension must be .cjs, not .cjs.js. Node.js would treat it as ESM if it were .js and package.json said type:module)
2. generate a dist .umd.js file.

package.json:

1. “type” is “module” (so Node.js will treat .js as .mjs rather than .cjs)
2. “main” must point to dist/name.cjs (so older versions of Node.js see CommonJS)
3. “module” must point to src/main.js (so tools like RESM use this instead of main)
4. “browser” must point to dist/name.umd.js (so tools like Rollup use this instead of main)
5. “exports.import” must point to src/main.js (so new Node.js versions use the ESM form)
6. “exports.require” must point to dist/name.cjs
7. “exports.browser” must point to dist/name.umd.js

This seems to cover all the bases.

[kriskowal]

Fixing with #217