Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Nested linear data structures (list, array) cause exponential size blowup

See original GitHub issue

Consider the code:

open FsCheck

Check.Quick(fun (t: int list list list list list) -> true)

Running the above never completes on my machine, I had to kill the process after it ended up allocating > 3 gigs of memory. Is this reasonable behaviour to expect or have I stumbled on some kind of performance bug?

Even though the type I gave is capable of having monstrous instances, my feeling is I should still be able to generate 100 instances without utterly exploding in size. Is this a reasonable expectation?

Issue Analytics

State:
Created 7 years ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

kurtschelfthoutcommented, Sep 21, 2016

I’m not sure what to call it (somewhere between feature and bug…), but I think I know what’s going on.

Size in relation to lists (and probably any linear data structre which is based on it) works like this. If you generate a list of size n, that means the length of the list can be up to n elements. So far so good. The problem occurs because this size is not “divided” among the elements, but each element is again generated with the same size bound (or maybe size-1, can’t remember, in any case not significantly lower).

So you generate a list with size 100, with each element being a list of size 100, and each of those again being a list of size 100 and so on. Exponential blowup.

A solution - perhaps - would be to instead divide the size among the elements somehow. So generate a list of size 100, then since it has 100 elements each of those has size 1 (or some other size that is less than 100). You see the problem: in your case, you’re only going to get empty lists at the leaves.

So it’s a bit tricky to tackle generically; in this case how the size is divided among constituents parts determines what the generator does.

Another way to look at it is that you’re really generating a 5-dimensional structure (like a 5D array). For that kind of structure, you probably want the number of elements to be proportional to size, whereas now it’s proportional to size^5. See the generator for Array2D for example.

If you’re interested, this issue is also described in https://gupea.ub.gu.se/handle/2077/22087

0reactions

kurtschelfthoutcommented, Sep 7, 2017

Released in 2.10.0.