`yarn rw test` Error: command doesn't run as of Apollo v3 upgrade

Hi 👋 , I have a problem with a new redwood project.

I run the following commands:

yarn create redwood-app test
cd test
yarn rw g component Test
yarn rw test

And I get the next error:

Jest encountered an unexpected token
...
    /home/rodrigo/workspace/test/node_modules/@apollo/client/react/hooks/useSubscription.js:1
    import { __assign } from "tslib";
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module
        at compileFunction (<anonymous>)

      at Runtime._execModule (../node_modules/jest-runtime/build/index.js:1179:56)

This is the output of yarn rw info

  System:
    OS: Linux 5.8 Arch Linux
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 12.18.3 - /tmp/yarn--1601732607280-0.7441083359073166/node
    Yarn: 1.22.10 - /tmp/yarn--1601732607280-0.7441083359073166/yarn
  Databases:
    SQLite: 3.33.0 - /usr/bin/sqlite3
  Browsers:
    Firefox: 81.0
  npmPackages:
    @redwoodjs/core: ^0.19.1 => 0.19.1 

Let me know if I can help somehow. Thanks in advance.

[aldonline]

We did some diggin’ with @thedavidprice and found the underlying cause. The solution, however, requires making some decisions that affect parts of the codebase that I don’t quite understand at this point. I can contribute some clarity and hope that this advances the discussion.

Here’s what’s going on:

• Apollo v3 introduced a significant change. They are now publishing a CommonJS (CJS) and an ES Modules (ESM) version of the library (they transpile to two targets and ship them to NPM)
  • This is good for tree shaking, for example (webpack can use the ESM version to include only what’s necessary)
  • But it means that there are two completely different set of *.js files shipped with their NPM package
  • The CJS files (the “traditional” files) end with *.cjs.js. Everything else is ESM.
  • They are following best practices, and including both a “main” and “module” field sin package.json, pointing to the CJS and ESM entrypoints respectively. So it is not entirely their fault.

So, if they are following best practices, why do our tests break? Because we’re doing some weird stuff that assumes a few things that are no longer true. This is hard to explain in the abstract, so I’ll just explain it step by step, starting from the error.

This is the error we get:

/private/tmp/twtestissue/testapp/node_modules/@apollo/client/react/hooks/useSubscription.js:1
    import { __assign } from "tslib";
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module
        at compileFunction (<anonymous>)

      at Runtime._execModule (../node_modules/jest-runtime/build/index.js:1179:56)
      at Object.<anonymous> (../node_modules/@redwoodjs/web/dist/index.js:78:24)
      at Object.<anonymous> (../node_modules/@redwoodjs/testing/dist/MockProviders.js:15:12)
      at Object.<anonymous> (../node_modules/@redwoodjs/testing/dist/customRender.js:17:22)
      at Object.<anonymous> (../node_modules/@redwoodjs/testing/dist/index.js:48:21)
      at Object.<anonymous> (../node_modules/@redwoodjs/core/dist/configs/browser/jest.setup.js:16:5)

• This all seems to start with browser/jest.setup.js.
  • To be more specific, at the top of the stack we have “some node process” that eventually runs jest.setup.js. The fact that there is a vanilla Node.js process at the top is important to keep in mind
• It then loads test/dist/index.js via require()
• A few more requires later, we are now executing web/dist/index.js
• which, in turn, tries to require the following:

var _useSubscription = require("@apollo/client/react/hooks/useSubscription")

• this require call will actually succeed in finding a file to load
• it follows the node resolution algorithm and ends up finding node_modules/@apollo/client/react/hooks/useSubscription.js
• however, this file is an ES Module (not a CommonJS module), so the parser fails when it finds an “import” keyword
  • Keep in mind that this is a vanilla “nodejs process” that is not running with any special flags to process ES Module syntax, or with any registered transpiler (like ts-node - which might be a patch solution to consider). It is just a traditional node process going about its day and loading CJS files.
  • Also, it doesn’t try to read package.json to find the “main” field (which is the CJS entry point) because our require specifier is “deep” (it goes beyond the top level “@apollo/client” and points directly to a folder “@apollo/client/react/hooks/…”)

At this point, we could fix this by changing web/dist/index.js so it requires the CJS version of the file, like so:

var _useSubscription = require("@apollo/client").useSubscription

And this will work (actually, we need to make a few changes to other parts of that file for that variable to be exported correctly, but that’s not important at this point).

Of course, we know web/dist/index.js comes from web/src/index.ts, so we should change this in the source file:

// this is how we're doing it today (following Apollo v3 instructions)
// but it compiles down to a require expression that node can't handle

// export { useSubscription } from '@apollo/client/react/hooks/useSubscription'
// export { useLazyQuery } from '@apollo/client/react/hooks/useLazyQuery'
// export { useQuery } from '@apollo/client/react/hooks/useQuery'
// export { useMutation } from '@apollo/client/react/hooks/useMutation'
// export { useApolloClient } from '@apollo/client/react/hooks/useApolloClient'

// so, instead, we could do it this way

export {
  useSubscription,
  useLazyQuery,
  useQuery,
  useMutation,
  useApolloClient,
} from '@apollo/client'

And this will now lead to a transpiled web/dist/index.js code that won’t break when the jest process tries to load it.

There might be a few issues with this approach (we need to validate this):

• We might not allow webpack to properly tree-shake the Apollo client when bundling the final redwood app (when webpack is bundling the final app it is using the “compiled” redwood framework in dist, so ESM-modularity is already lost).
• This file (web/dist/index.js) is used in several contexts: Testing, the actual app (and maybe storybook?). I don’t know enough to know if modifying it this way will break things

Now, there is one more place where Apollo imports must be changed. And this one is trickier:

web/src/graphql/withCell.tsx

import { Query } from '@apollo/client/react/components/Query'

Following the same logic above, we would need to replace it with something that, once transpiled, allows a Node.js process to correctly resolve it to a CJS file. For example, something like the following:

import { Query } from '@apollo/client'

Unfortunately, Apollo Client does not export “Query” at the top level. The closest CJS file in the package is: @apollo/client/react/components/components.cjs.js.

So the final (transpiled) code we need is something like this:

const Query = require("@apollo/client/react/components/components.cjs").Query

Which means that we would need to introduce some sort of try/catch or conditional check to know whether we import from @apollo/client/react/components/components.cjs or from @apollo/client/react/components/Query

Anyway. I can see ways of solving this, but I have some blind spots when it comes to the codebase and the different contexts in which files are loaded/transpiled/bundled, so at this point the best thing I can do is just share my findings.

My best “let’s get this fixed quickly” bet right now would be to teach the node process that ends up running Jest to handle ESM syntax when calling require() (this is different than the transpilation process that runs before Jest starts testing). Something like ts-node at the top-level might work.

[thedavidprice]

I thought @peterp mentioned to me during a conversation this week he could run the tests by removing all the Apollo Client imports. I’ll check. And I’m going to experiment with a few things now to see if I can isolate the error. There’s also a lot of related packages with upgrades available — possible we’re encountering a bug that’s been fixed.