question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Strongly increased memory consumption/waste since V2

See original GitHub issue

After having upgraded to Cloud SDK V2 and the corresponding VDMs, the memory consumption has increased, which can cause apps deployed to CloudFoundry to crash where they would have worked pretty well with SDK v1. In an export scenario where I fetch data in buckets from the SAP system, combine those and then pass the complete result as CSV to an endpoint, I compared using SDK/VDMs in V1.x to Version 2 (the VDM-APIs used here are mainly the ones from @sap/cloud-sdk-vdm-material-planning-data-service). The very same data was exported, the same machine and runtime was used. In both variants, the export took about 25 seconds, so no big difference in performance. The memory consumption looks as follows: Before V2 upgrade before-v2

After V2 upgrade after-v2

The heap consumption has a peak which is more than twice as high for V2 (600 MB) as it was in V1 (220 MB). External memory usage and ArrayBuffers show slightly higher values but the rate is not as significant as for the heap. Also a memory leak doesn’t apply in my mind as the garbage collector cleans up properly. Only the short-term heap usage seems to be a wasteful.

Used Versions:

  • node version: 14.18.2 (both scenarios)
  • npm version: 6.14.15 (both scenarios)
  • SAP Cloud SDK version: 1.54.0 / 2.1.0
  • VDM versions: 1.28.2 / 2.0.0

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:2
  • Comments:12 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
nocheintobicommented, Apr 7, 2022

Hi @jjtang1985, I gave it a try using executeRaw in my provided example and can confirm that memory consumption reduces heavily. Where my previous V2 sample consumed >800MB, using executeRaw took about 250MB at the peak.

Are there plans for optimizing the deserialization? If so, I’d prefer sticking to execute and working with the given types instead of switching the app code to using executeRaw

1reaction
jjtang1985commented, Apr 12, 2022

@nocheintobi ,

Thank you very much for your detailed reports 👍

We will plan a stable release for SDK cores (@sap-cloud-sdk/*) hopefully next week, including the generator. However, as I said, the vdm release @sap/cloud-sdk-vdm-material-planning-data-service was planned at the end of May. Let’s see whether we can make a new one before that. @marikaner , FYI.

Thank you very much once again for reporting this with professional approaches. We really appreciate it 😃

Best regards, Junjie

Read more comments on GitHub >

github_iconTop Results From Across the Web

Strongly increasing memory consumption when using ELMo ...
It worked, I still get an increase (from 1.5gb to 2gb), which I can not explain, but it seems much more managable. How...
Read more >
High Memory utilization and their root causes | Dynatrace
After all, more memory means longer garbage-collection suspensions. The following are the two most common causes for Java high memory usage.
Read more >
[Bug]: Memory consumption issues on Node JS 16.11.0+ ...
Since Jest calls global.gc() when Garbage Collector is exposed and --logHeapUsage flag is present, the memory usage should be stable.
Read more >
4 Types of Memory Leaks in JavaScript and How to Get Rid Of ...
Caches that grow unbounded can result in high memory consumption because their contents cannot be collected.
Read more >
Understanding memory consumption - Knowledge Base - Neo4j
While memory leaks can happen, more often than not, a higher memory consumption is normal behaviour by the JVM. In order to function...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found