Memory leak and performance issue
See original GitHub issueGenerating large file (> 10000 pages) crash node on my system (Linux Fedora 23) :
JavaScript heap out of memory
Here is my test script :
var PDFDocument = require('pdfkit');
var fs = require('fs');
var doc = new PDFDocument();
var v8 = require('v8');
var pageCount = 20000;
var getStats = function(text) {
var stats = v8.getHeapSpaceStatistics();
stats.forEach(function(stat) {
console.log(text + ' ' + stat.space_name + ' available size : ' + stat.space_available_size);
});
};
doc.pipe(fs.createWriteStream('test.pdf'));
console.time('Pdf generation time');
doc.on('end', function() {
console.log();
getStats('Final memory');
console.log();
console.timeEnd('Pdf generation time');
});
console.log('Start generating ' + pageCount + ' pages');
getStats('Starting memory');
for (var i = 1; i < pageCount; ++i) {
for (var y = 0; y < 40; ++y) {
doc.text('testText', y, y);
}
doc.addPage();
if (i % 4000 === 0) {
console.log();
getStats('Page ' + i);
}
}
doc.end();
And here is the result :
Start generating 20000 pages
Starting memory new_space available size : 775352
Starting memory old_space available size : 122488
Starting memory code_space available size : 231872
Starting memory map_space available size : 450104
Starting memory large_object_space available size : 1472650752
Page 4000 new_space available size : 8944344
Page 4000 old_space available size : 33672632
Page 4000 code_space available size : 1745920
Page 4000 map_space available size : 563520
Page 4000 large_object_space available size : 958758400
Page 8000 new_space available size : 9308184
Page 8000 old_space available size : 65610464
Page 8000 code_space available size : 1249696
Page 8000 map_space available size : 563432
Page 8000 large_object_space available size : 461798912
Page 12000 new_space available size : 1275960
Page 12000 old_space available size : 62591016
Page 12000 code_space available size : 782144
Page 12000 map_space available size : 564048
Page 12000 large_object_space available size : 0
<--- Last few GCs --->
[30138:0x34205a0] 218726 ms: Mark-sweep 1400.9 (1545.8) -> 1400.9 (1541.3) MB, 2200.4 / 0.1 ms (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 2201 ms) last resort
[30138:0x34205a0] 221096 ms: Mark-sweep 1400.9 (1541.3) -> 1400.9 (1541.3) MB, 2368.5 / 0.1 ms last resort
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x2313edd29891 <JS Object>
2: _text [/home/dev/svn/testPdfKit/node_modules/pdfkit/js/mixins/text.js:~33] [pc=0x9b80fbc8006](this=0x3c6fdb4a5a99 <a PDFDocument with map 0x1bebc6d107f9>,text=0x19e7d0cf6569 <String[8]: testText>,x=29,y=29,options=0x38aeea002311 <undefined>,lineCallback=0x69f67fdc639 <JS BoundFunction (BoundTargetFunction 0x3c6fdb4a9e31)>)
3: /* anonymous */ [/home/dev/svn/testPdfKit/pdfTest.js:~1] [...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [node]
2: 0x13647ec [node]
3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: v8::internal::Factory::NewByteArray(int, v8::internal::PretenureFlag) [node]
6: v8::internal::SourcePositionTableBuilder::ToSourcePositionTable(v8::internal::Isolate*, v8::internal::Handle<v8::internal::AbstractCode>) [node]
7: v8::internal::FullCodeGenerator::MakeCode(v8::internal::CompilationInfo*, unsigned long) [node]
8: v8::internal::FullCodegenCompilationJob::ExecuteJobImpl() [node]
9: v8::internal::CompilationJob::ExecuteJob() [node]
10: 0xd97d90 [node]
11: 0xd98b28 [node]
12: 0xd98d8f [node]
13: 0xd9ce59 [node]
14: v8::internal::Compiler::Compile(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag) [node]
15: v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*) [node]
16: 0x9b80f68437d
Abandon (core dumped)
After some analysis, i found out that the writable streams from referencePDF object eats a lot of memory. Same test result after removing the streams :
Start generating 20000 pages
Starting memory new_space available size : 2121376
Starting memory old_space available size : 600
Starting memory code_space available size : 352
Starting memory map_space available size : 450104
Starting memory large_object_space available size : 1472650752
Page 4000 new_space available size : 2915960
Page 4000 old_space available size : 21162392
Page 4000 code_space available size : 322848
Page 4000 map_space available size : 562200
Page 4000 large_object_space available size : 1162141184
Page 8000 new_space available size : 7767744
Page 8000 old_space available size : 69523024
Page 8000 code_space available size : 0
Page 8000 map_space available size : 562200
Page 8000 large_object_space available size : 877977088
Page 12000 new_space available size : 9225224
Page 12000 old_space available size : 101655856
Page 12000 code_space available size : 0
Page 12000 map_space available size : 562376
Page 12000 large_object_space available size : 589618688
Page 16000 new_space available size : 5595376
Page 16000 old_space available size : 133766616
Page 16000 code_space available size : 0
Page 16000 map_space available size : 562200
Page 16000 large_object_space available size : 301784576
Final memory new_space available size : 4856536
Final memory old_space available size : 569029752
Final memory code_space available size : 154112
Final memory map_space available size : 566336
Final memory large_object_space available size : 22240768
Pdf generation time: 82549.040ms
It seems that large_object_space
available memory gets eaten up a lot more slowly and the process also goes faster :
2000 pages with streams :
Start generating 2000 pages
Starting memory new_space available size : 775320
Starting memory old_space available size : 122472
Starting memory code_space available size : 234816
Starting memory map_space available size : 450104
Starting memory large_object_space available size : 1472650752
Final memory new_space available size : 8711240
Final memory old_space available size : 13663000
Final memory code_space available size : 285504
Final memory map_space available size : 561408
Final memory large_object_space available size : 1191419392
Pdf generation time: 5894.804ms
2000 pages without streams :
Start generating 2000 pages
Starting memory new_space available size : 9768096
Starting memory old_space available size : 428104
Starting memory code_space available size : 185024
Starting memory map_space available size : 152
Starting memory large_object_space available size : 1427037696
Final memory new_space available size : 13427376
Final memory old_space available size : 14697760
Final memory code_space available size : 1355936
Final memory map_space available size : 373800
Final memory large_object_space available size : 1289543168
Pdf generation time: 4895.007ms
Issue Analytics
- State:
- Created 6 years ago
- Comments:7 (1 by maintainers)
Top Results From Across the Web
Memory leak detection - How to find, eliminate, and avoid
A memory leak is any portion of an application which uses memory without eventually freeing it. By memory, we're talking about RAM, not ......
Read more >Memory Leaks and Garbage Collection | Computerworld
DEFINITION A memory leak is the gradual deterioration of system performance that occurs over time as the result of the fragmentation of a...
Read more >Memory leak - Wikipedia
In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations in...
Read more >How do I check for memory leaks, and what should I do to stop ...
The system can have a myriad of symptoms that point to a leak, though: decreased performance, a slowdown plus the inability to open...
Read more >How to Detect Memory Leaks in Java: Causes, Types, & Tools
Memory leaks block access to resources and cause an application to consume more memory over time, leading to degrading system performance. If ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Any plans on merging this? The memory leak also happens when generating a large number of smaller PDF files, causing continuous service restarts
If your code generates pages faster than they can be written to the stream, or if you want to limit worst-case memory consumption, wait for pageAdded events to fire. It is of course possible to alter this example to wait after X pages have been generated. You should also catch error events and invoke r() similarly.
Unfortunately, this example runs out of memory. I will create a new ticket.
Run this program using “node --max-old-space-size=64”