JavaScript heap out of memory on v12.0.1

Firstly, thanks for the package, it’s been working well for us so far.

We’re currently using v10.2.6 and experiencing an issue when trying to upgrade to v12.0.1. After upgrading the functionality still seems to be working as expected on our dev machines, but our tests have started to crash locally and on GitHub Actions with the following error:

<--- Last few GCs --->

[727712:0x558b398e0aa0]    72203 ms: Scavenge 2017.1 (2073.6) -> 2014.1 (2075.1) MB, 5.8 / 0.0 ms  (average mu = 0.831, current mu = 0.579) allocation failure
[727712:0x558b398e0aa0]    72295 ms: Scavenge 2018.6 (2075.1) -> 2016.3 (2078.4) MB, 6.3 / 0.0 ms  (average mu = 0.831, current mu = 0.579) allocation failure
[727712:0x558b398e0aa0]    72379 ms: Scavenge 2021.8 (2078.4) -> 2019.1 (2095.9) MB, 9.1 / 0.0 ms  (average mu = 0.831, current mu = 0.579) allocation failure


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
 1: 0x558b375212b1 node::Abort() [/usr/bin/node]
 2: 0x558b3743e144 node::FatalError(char const*, char const*) [/usr/bin/node]
 3: 0x558b376fb782 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/bin/node]
 4: 0x558b376fb9e8 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/bin/node]
 5: 0x558b378bb656  [/usr/bin/node]
 6: 0x558b378e7add v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [/usr/bin/node]
 7: 0x558b378f254e void v8::internal::LiveObjectVisitor::VisitBlackObjectsNoFail<v8::internal::EvacuateNewSpaceVisitor, v8::internal::MajorNonAtomicMarkingState>(v8::internal::MemoryChunk*, v
8::internal::MajorNonAtomicMarkingState*, v8::internal::EvacuateNewSpaceVisitor*, v8::internal::LiveObjectVisitor::IterationMode) [/usr/bin/node]
 8: 0x558b37912741 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::MemoryChunk*, long*) [/usr/bin/node]
 9: 0x558b378e2e25 v8::internal::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [/usr/bin/node]
10: 0x558b378e30f7 v8::internal::PageEvacuationTask::RunInParallel(v8::internal::ItemParallelJob::Task::Runner) [/usr/bin/node]
11: 0x558b378d4bc9 v8::internal::ItemParallelJob::Run() [/usr/bin/node]
12: 0x558b378f0169 void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactColle
ctor*, v8::internal::ItemParallelJob*, v8::internal::MigrationObserver*, long) [/usr/bin/node]
13: 0x558b37910c45 v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [/usr/bin/node]
14: 0x558b37910f59 v8::internal::MarkCompactCollector::Evacuate() [/usr/bin/node]
15: 0x558b379118f3 v8::internal::MarkCompactCollector::CollectGarbage() [/usr/bin/node]
16: 0x558b378c8e9f v8::internal::Heap::MarkCompact() [/usr/bin/node]
17: 0x558b378c98a8 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/bin/node]
18: 0x558b378c9ea7 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/bin/node]
19: 0x558b378cd54c v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/bin/node]
20: 0x558b378cd5b4 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/bin/node]
21: 0x558b378924be v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/bin/node]
22: 0x558b3788c5a2 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawArray(int, v8::internal::AllocationType) [/usr/bin/node]
23: 0x558b3788c654 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::int
ernal::AllocationType) [/usr/bin/node]
24: 0x558b37accfac v8::internal::OrderedHashTable<v8::internal::OrderedHashSet, 1>::Allocate(v8::internal::Isolate*, int, v8::internal::AllocationType) [/usr/bin/node]
25: 0x558b37acd17c v8::internal::OrderedHashTable<v8::internal::OrderedHashSet, 1>::Rehash(v8::internal::Isolate*, v8::internal::Handle<v8::internal::OrderedHashSet>, int) [/usr/bin/node]
26: 0x558b37acd7e1 v8::internal::OrderedHashTable<v8::internal::OrderedHashSet, 1>::Shrink(v8::internal::Isolate*, v8::internal::Handle<v8::internal::OrderedHashSet>) [/usr/bin/node]
27: 0x558b37bb8c66 v8::internal::Runtime_SetShrink(int, unsigned long*, v8::internal::Isolate*) [/usr/bin/node]
28: 0x558b37f33199  [/usr/bin/node]

This occurs after a test which usually takes the order of a few ms on v10.2.6 hangs for several seconds on v12.0.1. The memory usage of the app is usually low (~150MB) under normal operation, so I’m not sure what could be pushing it up so much. I don’t want to just increase the heap size without understanding the underlying cause.

I was wondering if you may be able to give me some pointers as to where I should start before I dive deep into trying to debug this? We’re using the namespace option during testing if that’s relevant in any way.

[cressie176]

Published as rascal@12.0.3. Thank you very much for your help tracking this one down, it really was some excellent work on your part.

[bausmeier]

I’ll see if I can extract some code which reproduces it to a public repo.