Memory leak and hang in histogram_bin_edges with strategy doane
See original GitHub issueFor the special case shown below, histogram_bin_edges with strategy doane hangs forever and wastes a single CPU and all system memory.
Reproducing code example:
Attention, will start to eat up all system memory!
import numpy as np
np.histogram_bin_edges([0.0, 0.0, 1.0], "doane", range=(0.0, 1056964608.0))
The error will not appear if only a single 0.0 is passed in the array.
Same thing happens with:
import numpy as np
np.histogram_bin_edges([0.0, 1.0, -1.0], "doane", range=(-1.0, 4294967296.0))
Error message:
No error message, code just hangs and wastes all memory.
Numpy/Python version information:
1.18.1 3.8.1 (default, Jan  8 2020, 23:09:20) 
[GCC 9.2.0]
the same thing happens with:
1.17.4 3.7.4 (default, Nov 21 2019, 11:30:33) 
[GCC 9.2.0]
(Issue found in a unit test using hypothesis)
Issue Analytics
- State:
 - Created 4 years ago
 - Comments:14 (8 by maintainers)
 
Top Results From Across the Web
CMS 12.3 is hanging due to memory leak - Optimizely World
The application is consuming all the memory in the server and hanging eventually. This is not happening on the developer's computer.
Read more >How we find and fix OOM and memory leaks in Java Services
1. Understanding OOM errors and identifying their causes · The application needs more memory than the OS can offer. · The Java application...
Read more >Memory Leaking Scenarios - Go 101
Real Memory Leaking Caused by Hanging Goroutines Such goroutines are called hanging goroutines. Go runtime will not kill hanging goroutines, so the resources ......
Read more >Finding Memory Leak in Go Service - Nylas
Golang's garbage collection strategy is the result of a design trade-off between CPU and memory usage. One round of garbage collection can take ......
Read more >Hunting Java Memory Leaks - Toptal
Our strategy for hunting down memory leaks will be relatively straightforward: Identify symptoms. Enable verbose garbage collection. Enable profiling. Analyze ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

Perhaps it would be possible to warn if it looks like things are not quite right? Not sure what the criterion for that would be.
Adding a
max_binsoption sounds like something that could also work for the other heuristics. For me it would be ok if the method simply raises an exception for cases where the number of generated bins would exceedmax_bins(before actually wasting the memory).