[Bug]: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Playwright version

1.13.0

Operating system

Linux

What browsers are you seeing the problem on?

Chromium

Other information

My environment:

Node v12.20.1
Python 3.8.10
Ubuntu 20.04

What happened? / Describe the bug

I created every domain url with new context and close context after get content and take a screenshot. When I ran the code more than 10.000 urls, the code likely crashed because of “JavaScript heap out of memory” error. I see some solutions to increase the memory limit in Node by setting NODE_OPTIONS=--max-old-space-size=<size> as an environment variable. So, I added export NODE_OPTIONS=--max-old-space-size=8192 in nano ~/.profile. But the setting didn’t fix the issue.

How can I fix the issue?

Thanks in advance,

Code snippet to reproduce your bug

context = await self.browser.new_context()
....
context.close()

Relevant log output

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa18150 node::Abort() [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 2: 0xa1855c node::OnFatalError(char const*, char const*) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 3: 0xb9715e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 4: 0xb974d9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 5: 0xd54755  [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 6: 0xd54de6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 7: 0xd616a5 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 8: 0xd62555 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
 9: 0xd6500c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
10: 0xd32eac v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
11: 0xe3e68b v8::internal::JsonParser<unsigned char>::MakeString(v8::internal::JsonString const&, v8::internal::Handle<v8::internal::String>) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
12: 0xe411a6 v8::internal::JsonParser<unsigned char>::ParseJsonValue() [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
13: 0xe4181f v8::internal::JsonParser<unsigned char>::ParseJson() [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
14: 0xc485fd v8::internal::Builtin_JsonParse(int, unsigned long*, v8::internal::Isolate*) [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
15: 0x140a9f9  [/home/ubuntu/.local/lib/python3.8/site-packages/playwright/driver/node]
Aborted (core dumped)

[phongtnit]

How many URLs are you navigating to per context? Do you have a reproducible?

Hi @mxschmitt , my script created one unique context for one url and close the context after crawling the url. It worked fine about first 10k urls. However, when the script ran more than 10k urls, the error occurs, sometimes, the error occurred while accessing about 100k urls. I guess the issue was about release memory in Nodejs.

[phongtnit]

@mxschmitt No, I used uuid4() to create new unique name of context with every urls.