question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Out of memory when running convert_dir_to_note_sequences

See original GitHub issue

Hello everyone!

Today I observed something strange when using convert_dir_to_note_sequences.

When I run the tool on the Lakh dataset I observe this:

  1. The script runs fine for a while. Converting one MIDI file after the other.
  2. While the script is running the RAM usage is very low. I monitor with the tool free on Unix.
  3. The script suddenly freezes. It stops at a file.
  4. Looking at free shows that the memory usage quickly goes up until all memory is exhausted.
  5. After a while the script terminates with the message Killed.

I have 60GB of RAM on the system. Magenta version is 2.1.0. Tensorflow version is 2.2.0.

I added a little logging to the script to get the file that is currently being processed. This is what I get:

INFO:tensorflow:Attempting to convert MIDI file lmd_full_split/train/a/a79fe18e50fb30cd1679e3a9e0960620.mid.
I0623 14:55:44.641093 140074287511360 convert_dir_to_note_sequences.py:141] 

Killed

I checked the file in Logic Pro X. It seems to work.

Any ideas?

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
iljabcommented, Dec 14, 2020

@AmitMY @AI-Guru Same problem here, do you have any hints?

0reactions
AmitMYcommented, Sep 24, 2020

@AI-Guru I have the same problem. Did you manage to convert lakh dataset to tfrecord? If so, could you please share how?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Running out of memory - IBM
A very common difficulty with MIPs is running out of memory. This problem almost always occurs when the branch & cut tree becomes...
Read more >
Out-of-memory error when you run an instance of SQL Server ...
Fixes an issue in which you receive an out-of-memory error when you run SQL Server 2012 on a computer that uses NUMA. This...
Read more >
vite build error: out of memory · Issue #2433 - GitHub
I discovered that it only happens when I rm -r node_modules , npm install and then npm run dev . If I work...
Read more >
Out-of-Memory (OOM) or Excessive Memory Usage
When a node crashes, the OSC staff has to manually reboot and clean up the node. If other jobs were running on the...
Read more >
Error: "Out of memory" - Knowledge Base - COMSOL
If you anticipate running many models of similar size, then it is reasonable to consider a hardware upgrade. See Knowledge Base solution 866...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found