question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Memory leak and hang in histogram_bin_edges with strategy doane

See original GitHub issue

For the special case shown below, histogram_bin_edges with strategy doane hangs forever and wastes a single CPU and all system memory.

Reproducing code example:

Attention, will start to eat up all system memory!

import numpy as np
np.histogram_bin_edges([0.0, 0.0, 1.0], "doane", range=(0.0, 1056964608.0))

The error will not appear if only a single 0.0 is passed in the array.

Same thing happens with:

import numpy as np
np.histogram_bin_edges([0.0, 1.0, -1.0], "doane", range=(-1.0, 4294967296.0))

Error message:

No error message, code just hangs and wastes all memory.

Numpy/Python version information:

1.18.1 3.8.1 (default, Jan  8 2020, 23:09:20) 
[GCC 9.2.0]

the same thing happens with:

1.17.4 3.7.4 (default, Nov 21 2019, 11:30:33) 
[GCC 9.2.0]

(Issue found in a unit test using hypothesis)

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:14 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
charriscommented, Jan 15, 2020

Perhaps it would be possible to warn if it looks like things are not quite right? Not sure what the criterion for that would be.

0reactions
languitarcommented, Jan 16, 2020

Adding a max_bins option sounds like something that could also work for the other heuristics. For me it would be ok if the method simply raises an exception for cases where the number of generated bins would exceed max_bins (before actually wasting the memory).

Read more comments on GitHub >

github_iconTop Results From Across the Web

CMS 12.3 is hanging due to memory leak - Optimizely World
The application is consuming all the memory in the server and hanging eventually. This is not happening on the developer's computer.
Read more >
How we find and fix OOM and memory leaks in Java Services
1. Understanding OOM errors and identifying their causes · The application needs more memory than the OS can offer. · The Java application...
Read more >
Memory Leaking Scenarios - Go 101
Real Memory Leaking Caused by Hanging Goroutines​​ Such goroutines are called hanging goroutines. Go runtime will not kill hanging goroutines, so the resources ......
Read more >
Finding Memory Leak in Go Service - Nylas
Golang's garbage collection strategy is the result of a design trade-off between CPU and memory usage. One round of garbage collection can take ......
Read more >
Hunting Java Memory Leaks - Toptal
Our strategy for hunting down memory leaks will be relatively straightforward: Identify symptoms. Enable verbose garbage collection. Enable profiling. Analyze ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found