JavaScript heap out of memory after upgrade to Jest 26

🐛 Bug Report

I upgraded from 24.X to 26.0.0 but now test that was passing is not Running test takes long time to complete then I get this error

To Reproduce

My test:

  describe('when item ids are in sessionStorage', () => {
    const itemIds = [333, 222, 111];

    beforeEach(() => {
      parseLocationToQueries.mockImplementation(() => ({
        queue_id: testQueueId
      }));
      isAdHocReviewByItemId.mockReturnValue(false);
      isAdHocReviewByObjId.mockReturnValue(false);
      setItemsToBeReviewed(itemIds);
    });

    it('initial fetch', () => {
      const wrapper = tf.render();
      expect(wrapper.state('itemIds')).toEqual([]);
      expect(axios.post).toBeCalledWith('/review/items', { item_ids: itemIds });
    });

    it('fetch more while no more', () => {
      const wrapper = tf.render();
      axios.post.mockClear();
      wrapper.instance().fetchMoreItems();
      expect(axios.post).not.toBeCalled();
    });

    it('fetch more while more', () => {
      const wrapper = tf.render();
      axios.post.mockClear();
      wrapper.setState({ itemIds: [555] });
      wrapper.instance().fetchMoreItems();
      expect(axios.post).toBeCalledWith('/review/items', { item_ids: [555] });
    });
  });

code:

export function setItemsToBeReviewed(itemIds) {
  sessionStorage.setItem(ITEMS_TO_BE_REVIEWED_KEY, JSON.stringify(itemIds));
}


  fetchMoreItems = () => {
    this.setState({ loadingMoreItems: true });
    return this.fetchItems(true)
      .then(res => {
        this.loadData(res.data);
      })
      .catch(error => {
        console.log('FetchmoreError', error);
      });
  };

  fetchItems = (excludeAssigned: boolean = false) => {
    let request;
    if (this.state.itemIds) {
      request = this.fetchItemsByIds();
    } else {
      request = this.fetchItemsFIFO(excludeAssigned);
    }
    return request;
  };

  fetchItemsFIFO = (excludeAssigned: boolean = false) => {
    const { isAlignment, queueIdFromURL } = this.state;
    const url = '/review/assign';
    const params = {
      alignment: isAlignment,
      queue_id: queueIdFromURL,
      exclude_assigned: excludeAssigned
    };
    return axios.get<any>(url, { params });
  };

  fetchItemsByIds = () => {
    if (_.isEmpty(this.state.itemIds)) {
      return Promise.resolve({ data: [] });
    }
    const url = '/review/items';
    const data = {
      item_ids: _.slice(this.state.itemIds, 0, FETCH_BATCH_SIZE)
    };
    this.setState(state => ({
      itemIds: _.slice(state.itemIds, FETCH_BATCH_SIZE)
    }));
    return axios.post<any, any>(url, data);
  };

jest.config:

module.exports = {
  timers: 'fake',
  moduleDirectories: ['node_modules'],
  moduleFileExtensions: ['js', 'jsx'],
  moduleNameMapper: {
    '\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$':
      '<rootDir>/__mocks__/fileMock.js',
    '\\.(css|less)$': '<rootDir>/__mocks__/styleMock.js',
    '^Root(.*)$': '<rootDir>$1',
    '^Utils(.*)$': '<rootDir>/src/utils$1',
    '^Hoc(.*)$': '<rootDir>/src/hoc$1',
    '^Components(.*)$': '<rootDir>/src/components$1'
  },
  testRegex: 'test\\.jsx?$',
  testURL: 'http://localhost:3000',
  collectCoverageFrom: [
    'src/**/*.js',
    'src/**/*.jsx',
    '!**/node_modules/**',
    '!src/components/bulk_review/columns/**',
    '!src/components/v2/**'
  ],
  coverageReporters: ['html', 'text'],
  coverageThreshold: {
    global: {
      branches: 90,
      functions: 90,
      lines: 90,
      statements: 90
    }
  },
  coverageDirectory: 'coverage',
  snapshotSerializers: ['enzyme-to-json/serializer'],
  testEnvironment: '<rootDir>/jest-environment.js',
  setupFilesAfterEnv: ['<rootDir>/enzyme.setup.js'],
  setupFiles: [
    '<rootDir>/__mocks__/localStorageMock.js',
    '<rootDir>/__mocks__/consoleMock.js'
  ],
  globals: {
    ENVIRONMENT: 'TESTING'
  },
  testPathIgnorePatterns: ['<rootDir>/src/components/v2'],
  reporters: [
    'default',
    [
      'jest-html-reporter',
      {
        pageTitle: 'Test Report',
        statusIgnoreFilter: 'passed',
        includeFailureMsg: 'true'
      }
    ]
  ]
};

envinfo

System: OS: Linux 4.15 Ubuntu 18.04.4 LTS (Bionic Beaver) CPU: (36) x64 Intel® Xeon® Platinum 8124M CPU @ 3.00GHz Binaries: Node: 14.1.0 - ~/.nvm/versions/node/v14.1.0/bin/node Yarn: 1.22.4 - /usr/bin/yarn npm: 6.14.4 - ~/.nvm/versions/node/v14.1.0/bin/npm npmPackages: jest: ^26.0.0 => 26.0.0

[alexfromapex]

After doing some research, it seems this memory leak has been an ongoing issue since 2019 (Jest 22) so wanted to consolidate some notes here for posterity. Past issues have been related to graceful-fs and I think some have solved it via a hack/workaround that removes graceful-fs and then re-adds graceful-js after running jest. One troubleshooting thread was looking at compileFunction in the vm package as a potential cause. It seems that jest, webpack-dev-server, babel, and create-react-app are using graceful-js as a dependency. The memory leak issue was supposed to be fixed in a newer release of Jest but there may have been a regression since it is popping up again. I can confirm everything was working fine until a substantial amount of Jest tests were created in our environment and then the heap overflows on our CI machine after the heap size grows larger than the allocated memory due to the leak. I’ve tried using 1 worker, runInBand, etc. without success.

The common cause of the issues I’ve seen is collecting coverage via collecting coverage and graceful-fs. I haven’t done an in-depth analysis of those issues but seeing that they are both filesystem-related and having solved my own issue which was related to file imports I suspect they are some version of the same issue I was having.

Wanted to provide the solution I found so others may reap benefits:

The cause:

Using imports of the format import * from 'whatever'

The solution:

Using the format import { whatINeed } from 'whatever' instead dramatically reduced the memory accumulation

[mike-4040]

Trying to migrate from 26.6.3 to 27.2.5 and got the same issue.