Heap error gets thrown regardless of include/exclude - Process memory threshold reached.

Bug

Current Behavior

I used to be on the cli version 0.12.3 and not run into any issues, but after upgrading recently to 0.12.6 I have started seeing the following error whenever I try to bring up my garden dev in local.

Process memory threshold reached. This most likely means there are too many files in the project, and that you need to exclude large dependency directories. Please see https://docs.garden.io/using-garden/configuration-overview#including-excluding-files-and-directories for information on how to do that.
If this keeps occurring after configuring exclusions, please file an issue at https://github.com/garden-io/garden/issues.

I’ve tried adding include: [] to all my modules in the project but still seem to run into the issue. I am thinking it’s probably due to the pure size of the project I am trying to bring up – and I happen to fall in the 85%-100% range heap usage range. My project already respects the ignore files also -

dotIgnoreFiles:
  - .gardenignore
  - .gitignore

I’ve tried setting an env var via export NODE_OPTIONS=--max_old_space_size=4096 to increase node’s max memory limit but that somehow break garden altogether where i can’t even do garden --version.

Expected behavior

• There should be an option to increase the max heap size limit in case it was ever needed.
• The error message should be more straightforward – more details in the Workaround section.

Reproducible example

1. export NODE_OPTIONS=--max_old_space_size=4096 in the shell
2. Run garden --version

Expected: version output

Actual: nothing

Workaround

1. Downgrading manually to 0.12.3 version that did not have the heap check. (may only work if your heap usage falls between 85% but still not using up all 100%)
2. I’ve found out that by removing certain duplicate dependencies e.g., if my module F had dependencies like

dependencies:
- A
- B
- C

and B happened to depend on A already, and C happened to depend on B, then just depending on C seems to relieve the issue for me. Perhaps, this is the real culprit of the issue – my listing out all dependencies in my project was trying to make it more apparent what the module relied on.

Suggested solution(s)

• It seems like the heap error could possibly result from duplicated (inferred) dependencies being listed in some modules as opposed to what the error message indicates – e.g., include/exclude files.
• Allow for increasing the max heap for the garden engine.

Your environment

• OS: macOS
• How I’m running Kubernetes: docker

garden version - 0.12.6

[edvald]

Aha, I see, thanks! I think that narrows it down for sure.

[thsig]

@kyungkimsf Thanks for flagging this! We’ll look into it.