Jest fails to process a file inside `node_modules`, giving error `Cannot use import statement outside a module`

Verify canary release

• I verified that the issue exists in Next.js canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000
Binaries:
  Node: 16.13.1
  npm: 8.1.4
  Yarn: 1.22.17
  pnpm: N/A
Relevant packages:
  next: 12.1.5-canary.4
  react: 17.0.2
  react-dom: 17.0.2

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

I’m using the module remark in one of my components that I need to test. But when I run the test for that component with Jest. I see the following error

Jest encountered an unexpected token

Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

By default "node_modules" folder is ignored by transformers.

Here's what you can do:
  • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
  • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
  • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
  • If you need a custom transformation specify a "transform" option in your config.
  • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/configuration
For information about custom transformations, see:
https://jestjs.io/docs/code-transformation

Details:

/Users/mk/Code/github/hollowverse/hollowverse/node_modules/remark/index.js:1
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import {unified} from 'unified'
                                                                                  ^^^^^^

SyntaxError: Cannot use import statement outside a module

  1 | import matter from 'gray-matter';
> 2 | import { remark } from 'remark';
    |                                ^
  3 | import remarkHtml from 'remark-html';
  4 | import { sanityClient } from '~/lib/components/sanityio';
  5 | import { groqRelatedPeople } from '~/lib/[celeb]/getStaticProps/groqRelatedPeople';

  at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1728:14)
  at Object.<anonymous> (lib/[celeb]/getStaticProps/getParsedOldContent.ts:2:32)

My jest.config.js is the following

// jest.config.js
const tsconfig = require('./tsconfig.json');
const moduleNameMapper = require('tsconfig-paths-jest')(tsconfig);
const nextJest = require('next/jest');

const createJestConfig = nextJest({
  // Provide the path to your Next.js app to load next.config.js and .env files in your test environment
  dir: './',
});

// Add any custom config to be passed to Jest
const customJestConfig = {
  // Add more setup options before each test is run
  // setupFilesAfterEnv: ['<rootDir>/jest.setup.js'],
  // if using TypeScript with a baseUrl set to the root directory then you need the below for alias' to work
  moduleDirectories: ['node_modules', '<rootDir>/'],
  testEnvironment: 'jest-environment-jsdom',
  moduleNameMapper: {
    ...moduleNameMapper,
    '^lodash-es$': 'lodash',
  },
};

// createJestConfig is exported this way to ensure that next/jest can load the Next.js config which is async
module.exports = createJestConfig(customJestConfig);

Expected Behavior

I expect next/jest to handle this situation?

To Reproduce

Follow the recommended steps to setup a Next.js project with Jest support, then build a component that uses the remark module and write a test for it.

[burtek]

For now I went with

- module.exports = createJestConfig(customJestConfig);
+ module.exports = async () => ({
+     ...await createJestConfig(customJestConfig)(),
+     transformIgnorePatterns: [
+         'node_modules/(?!(monaco-editor)/)',
+         '^.+\\.module\\.(css|sass|scss)$',
+     ]
});

Should work for @msafi as workaround for now, just replace monaco-editor with remark. More libs can be added alongside, |-separated (i.e. 'node_modules/(?!(lib1|lib2|lib3)/)'

transformIgnorePatterns must be added outside of createJestConfig as the latter overrides the provided transformIgnorePatterns in case of node_modules

[burtek]

@msafi Had the same happen to me when I forgot the () at the end of await createJestConfig(customJestConfig)() - maybe that’s the issue?

Also, only transformIgnorePatterns needs to be outside createJestConfig, the rest can go in customJestConfig - it should work fine.