Partition methods unsound on infinite iterables

There are some methods which require arrays to work correctly. Sort is an obvious example. I am going to posit that partition and multiPartition are actually another.

We already know that methods which must operate on arrays are outside the scope of iter-tools. This is primarily because we mean to support all iterables, including infinite iterables, which cannot ever be cached in arrays. This, however, is not the only pitfall of infinite iterables. Another can be easily demonstrated here:

const [winners, alsoRan] = partition(i => i < 3, range())
for (const winner of winners) {
   /* Oops, we created an infinite loop.
    * We will continue trying to pull from the infinite source iterable
    * looking for the next value less than three. There is none.
    */
}

Essentially this means that if you have an infinite iterable, the method will only work well if you are sure that one type of object will never cease appearing in the output.

Consider though the case of the async variants of these methods when they are simply run on large iterables. It would almost always be the case that it would be inappropriate to try to partition the iterables using these methods. Trying to consume any one side of the partition in its entirety would be no faster than consuming the whole source. If I were trying to write code that was responsive to an async iterable of alphas and betas, I’d do something like const [alphas, betas] = partition(isAlpha, source). Then in order to avoid being blocked, I’d have to try to consume the alphas and betas iterables simultaneously, using something like Promise.race to determine which result was actually ready. Essentially at this point the coder would be working defeat the implementation of our method, which mostly has to do with caching items, a behavior that seems to actually be undesired the more I think about it.

I think most of the time the appropriate thing to do in the code is simply

for await (const item of source) {
  if (isAlpha(item)) {
    // Whatever is supposed to happen for an alpha
  } else {
    // Whatever is supposed to happen to beta
  }
}

Can anyone imagine a situation in which it would be more suitable to use partition than to just write a loop like the one above?

[sithmel]

Using an infinite iterable we can have basically 2 potential side effect:

• some method will just blow up (because of memory). toArray and takeSorted for example
• some other will run forever and never return

From a certain point of view, it should be up to the dev to ensure that this is not happening (“slice” would do the trick easily).

Technically if we don’t implement that it’s a security vulnerability.

I am not sure you could define that as a security vulnerability. It would be like saying that you have to remove the while loop from the language because it can be leveraged to run a DOS.

[conartist6]

The issue partition has over and beyond other similar methods like groupBy is that with groupBy as long as you consume the all the result iterables in order you can be assured that your code will not end up in an infinite loop. With partition, the same consumption strategy will sometimes result in an infinite loop, and sometimes not. In my mind this means that the appropriate thing to do is use lodash: const [evens, odds] = _.partition([...iterable], isEven);.