Out of memory when running convert_dir_to_note_sequences
See original GitHub issueHello everyone!
Today I observed something strange when using convert_dir_to_note_sequences
.
When I run the tool on the Lakh dataset I observe this:
- The script runs fine for a while. Converting one MIDI file after the other.
- While the script is running the RAM usage is very low. I monitor with the tool
free
on Unix. - The script suddenly freezes. It stops at a file.
- Looking at
free
shows that the memory usage quickly goes up until all memory is exhausted. - After a while the script terminates with the message
Killed
.
I have 60GB of RAM on the system. Magenta version is 2.1.0. Tensorflow version is 2.2.0.
I added a little logging to the script to get the file that is currently being processed. This is what I get:
INFO:tensorflow:Attempting to convert MIDI file lmd_full_split/train/a/a79fe18e50fb30cd1679e3a9e0960620.mid.
I0623 14:55:44.641093 140074287511360 convert_dir_to_note_sequences.py:141]
Killed
I checked the file in Logic Pro X. It seems to work.
Any ideas?
Issue Analytics
- State:
- Created 3 years ago
- Comments:5
Top Results From Across the Web
Running out of memory - IBM
A very common difficulty with MIPs is running out of memory. This problem almost always occurs when the branch & cut tree becomes...
Read more >Out-of-memory error when you run an instance of SQL Server ...
Fixes an issue in which you receive an out-of-memory error when you run SQL Server 2012 on a computer that uses NUMA. This...
Read more >vite build error: out of memory · Issue #2433 - GitHub
I discovered that it only happens when I rm -r node_modules , npm install and then npm run dev . If I work...
Read more >Out-of-Memory (OOM) or Excessive Memory Usage
When a node crashes, the OSC staff has to manually reboot and clean up the node. If other jobs were running on the...
Read more >Error: "Out of memory" - Knowledge Base - COMSOL
If you anticipate running many models of similar size, then it is reasonable to consider a hardware upgrade. See Knowledge Base solution 866...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@AmitMY @AI-Guru Same problem here, do you have any hints?
@AI-Guru I have the same problem. Did you manage to convert lakh dataset to tfrecord? If so, could you please share how?