question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"Fork" should fire an exception when trying to consume inner iterables without consuming the container one

See original GitHub issue

Using fork in a for … of loop is highly wasteful because it forces to store all input iterable in memory.

for (const f of fork(inputIterable)) {
  // whatever I do with f here will be cached in the buffer.
  // I can't consume the buffer because I am still able to create a new fork
  // and that will return the items from the beginning
}

The sensible way to use fork is using destructuring:

const [f1, f2] =  fork(inputIterable)

For this reason I think it would be better to forbid to consume the forked iterables when the original iterable is still able to emit new “clones”.

for (const f of fork(inputIterable)) {
  f[Symbol.iterator]().next() // Error! You can't consume the forked iterable while you can still emit new iterables
}

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:1
  • Comments:10 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
conartist6commented, Feb 14, 2019

I’m against this restriction. I considered it as a design option when I wrote my implementation (and voiced that possibility). I guess I hadn’t fully decided, just put the decision off for later.

But let’s look at the code you’re afraid of:

for (const f of fork(inputIterable)) {
  // ...
}

I don’t think this code is really any cause for worry. First, the code is an infinite loop. If there’s a break statement then we get to do our cleanup. If there’s no break statement, then literally the whole rest of the program must be about this iterable, in which case the cache hardly seems like a leak or waste of memory.

I’d argue that you’re also confusing our design intentions a bit. We put in the ability to discard unused cache items as an optimization for a specific use case, when the forks were ultimately going to be reintegrated item by item with zip, merge, or collate. The other use cause, however, where the user consumes some or all of each fork in sequence, is also perfectly valid and probably common. We don’t need to fear that our optimization won’t apply there, we just know that it wasn’t meant to, because with that access pattern it doesn’t really matter if we can or can’t make more forks. If we can’t make more forks, there must be some number n of forks that we made. With sequential access the cache won’t be cleaned up at all until we’re already iterating over the nth fork anyway, which is not much of an optimization.

0reactions
conartist6commented, Feb 14, 2019

Yeah we should certainly express in documentation that there is reason to consider using Array.from() or toArray instead of fork, and that you need to be quite careful it you’re using a fork around infinite iterables

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why don't empty iterables in python raise Exceptions when ...
If you try to iterate over a generator with next and it's empty or has reached the end, a StopIteration exception is raised,...
Read more >
How yield catches StopIteration exception? - Stack Overflow
A generator function will never finish iterating without raising StopIteration (unless it raises some other exception instead).
Read more >
How can i access the exception that fire handles? #209 - GitHub
I know that whether I am using a class or a function, fire will internally handle the exception and show the help if...
Read more >
Python Stacks, Queues, and Priority Queues in Practice
In this tutorial, you'll take a deep dive into the theory and practice of queues in programming. Along the way, you'll get to...
Read more >
Groovy Language Documentation
An embedded closure expression taking more than one parameter will generate an exception at runtime. Only closures with zero or one ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found