CircleCI Crashes on Ubuntu instances due to Memory Usage

Ask your Question

The CircleCI boxes that are used for ubuntu testing (specifically our android build) have 36 cores. When we are running our yarn bundle --platform=android build, we are running out of memory with this error:

./node_modules/react-native/Libraries/Utilities/stringifySafe.js
Module build failed (from ./node_modules/haul/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 32)
ENOMEM: not enough memory, read

We don’t see this problem on our other workstations, that have 8 cores.

When we modified the thread-loader options to only have 1 worker, we no longer see this issue.

How do you recommend that we modify the configuration to have less thread-workers? What strategies do people normally use to merge in webpack config settings? webpack-merge?

Currently we are doing this:

export default {
  webpack: env => {
    const config = createWebpackConfig({
      entry: `./index.js`
    })(env)
    config.module.rules[1].use[1].options.workers = 1
    // A bunch of other rules and settings
  }
}

But this is obviously sub-optimal.

Thanks for looking and helping!

James

[hjhart]

Yeah, I can submit a pull request. Much friendlier than manipulating nested JavaScript objects…

On Apr 6, 2019, at 16:28, Tommy Nguyen [e-mail] wrote:

Ah, I completely missed the number of cores. Sorry about that. The thread count is set in src/utils/makeReactNativeConfig.js:

{ loader: require.resolve(‘thread-loader’), options: { workers: Math.max(os.cpus().length - 1, 1), }, }, It looks like we’re counting logical cores and subtract one. Should be simple to make configurable. What do you think, @thymikee? @hjhart, would you like to give it a try?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[tido64]

I seem to recall reading something about Webpack having memory leaks. Which version are you using? Could you try upgrading and see if that helps?