Silent crash when trying to set large numpy array
See original GitHub issueWhen I’m trying to set some large numpy array variables in Matlab through the set_variable method, my Matlab server close and the notebook stays idling forever. Here is a minimal code for reproducing this error (I’m running it in a notebook):
import numpy as np
from pymatbridge import Matlab
import sys
mlab = Matlab(maxtime=60) #Matlab(matlab='C:/Program Files/MATLAB/R2015a/bin/matlab.exe')
mlab.start()
print("Before")
sys.stdout.flush()
mlab.set_variable("in0", np.zeros((3814400,1)))
print("After")
sys.stdout.flush()
This gives me the output:
Starting MATLAB on ZMQ socket tcp://127.0.0.1:51192
Send 'exit' command to kill the server
............MATLAB started and connected!
Before
And at that point, the Matlab server window close automatically, “After” never get printed and the notebook stays idling (i.e. it shows “In [*]” forever) forcing me to reboot the kernel.
Issue Analytics
- State:
- Created 8 years ago
- Comments:13 (4 by maintainers)
Top Results From Across the Web
Python (numpy) crashes system with large number of array ...
This seems like a case of stack overflow. You have 3,682,800,000 array elements, if I understand your question. What is the element type?...
Read more >Dying, fast and slow: out-of-memory crashes in Python
Crashing is just one symptom of running out of memory. Your process might instead just run very slowly, your computer or VM might...
Read more >Broadcasting — NumPy v1.24 Manual
When the trailing dimensions of the arrays are unequal, broadcasting fails because it is impossible to align the values in the rows of...
Read more >Size limit to NumPy Arrays? - Esri Community
I've got my script working the way I want. Now when I try and run it with my full extent raster it crashes...
Read more >How to avoid a Pandas pandemonium, Part I | by Pamela Wu
Part I: Writing good code and spotting silent failures. ... I'll make a large DataFrame to illustrate larger processing problems and a small ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I ran into the same problem while passing around some larger numpy arrays. I’m working with a 64bit version of Matlab R2013a on Linux Mint 14.1 (IcedTea Java - I don’t know the precise version). To some extent you can stop this from happening if you increase the Java Heap Memory, that Matlab is allowed to use.
For further information on this topic and an instruction, see the Matlab documentation here and an older Mathworks blog post here.
I’m sorry: I am not really using Matlab much these days, and have neither the bandwidth, nor the strong motivation to dig into these issues. I suspect that it might have to do with hard-coded limitations (maybe this? https://github.com/arokem/python-matlab-bridge/blob/master/pymatbridge/messenger/src/messenger.c#L8)