question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Suggestion: Function that turns an iterator into an iterable that lasts forever.

See original GitHub issue

Usage

const iterator = range(4)
const iterable = everlasting(iterator)
Array.from(iterable) // [0, 1, 2, 3]
Array.from(iterable) // [0, 1, 2, 3]
Array.from(iterator) // []
Array.from(iterable) // [0, 1, 2, 3]
let count = 0

function * generator () {
  yield count++
  yield count++
  yield count++
}

const iterator = generator()
const iterable = everlasting(iterator)

count // 0
Array.from(count) // [0, 1, 2]
count // 3

Features

  • The returning iterable (iterable) is lazy, i.e. input iterable (iterator) won’t be consumed until the returning iterable is consumed for the first time.
  • After fully exhausted for the first time, it still emits elements (from an internal cache).

Suggestion

Either:

  1. Change existing iterable and asyncIterable functions
  2. Create new functions

Alternatives

Why not Array.from, toArray, asyncToArray? ref

Arrays are not lazy.

Why not tee/fork? ref

It is too verbose.

I would rather not have tee/fork as everlasting can do what tee/fork can do and more:

  • everlasting can clone itself, that’s what tee/fork can do.
  • everlasting’s usage is simpler, that’s what tee/fork can’t do.
  • everlasting’s implementation is simpler, that’s what tee/fork refused to do.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:22 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
conartist6commented, Feb 5, 2019

@KSXGitHub I think you have a fundamental misunderstanding.return is pretty important. First, since you mentioned it, return is not called when an iterable is exhausted. Iterables can return two ways. They can inform their consumer that they have run out of items, in which case they initiate any required cleanup, or their consumers can inform them that they no longer intend to consume any more items, in which case return is called to give the iterator a chance to do its cleanup.

The thing is, in order to know that it is safe to call return on the source, you must know for sure that you will not need any more of it. Your proposed method could not possibly know that, because at any time someone could call Symbol.iterator and the method would need to produce an iterator capable of iterating all the way to the end of the source. This doesn’t matter if the iterable is exhausted completely, which is the only case you seem to be considering. It does matter if the iterable is not exhausted, such as because somebody called slice() on it.

Now sometimes this might not matter, because ultimately the garbage collector will come around and clean up the whole thing, but sometimes there is more going on than memory. Consider the following iterable:

function * readLinesFromFile(file) {
   const fileDescriptor = fs.openSync(file)
   try {
     ...
   } finally {
     fs.closeSync(fileDescriptor)
   }
}

Lets say you used this function and called slice on it in order to implement something like the head command. If you were to call everlasting here, you would leak a file handle. This is just one instance of code doing something that requires cleaning up. Another might be registering an event listener (which would even prevent the GC from helping). It is not worth, creating all these largely invisible problems just to simplify the API a bit. The simple API would not be simpler when it would need to be attached to a lengthy documentation of disclaimers documenting all the situations in which it would be unsafe.

And finally, I repeat: most of the time what you want will simply be accomplished by calling Array.from(iterable)

1reaction
conartist6commented, Feb 5, 2019

Haha glad I could help. It’s complicated, and I don’t think any of us understood it properly before we started writing code in this repo (and discovering the ways in which it was incorrect)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Is there any built-in way to get the length of an iterable in ...
Short of iterating through the iterable and counting the number of iterations, no. That's what makes it an iterable and not a list....
Read more >
Convert Iterator to Iterable in Java - GeeksforGeeks
Java program to get a Iterable. // from a given Iterator. import java.util.*;. class GFG {. // Function to get the Spliterator.
Read more >
pytorch/dataloader.py at master - GitHub
Tensors and Dynamic neural networks in Python with strong GPU acceleration - pytorch/dataloader.py at master · pytorch/pytorch.
Read more >
Beginners Guide to Iterators in Python - Edureka
To stop an Iteration from going on forever, we use the StopIteration statement. Let us understand this with a few examples. 1. 2....
Read more >
A Study of Python's More Advanced Features Part I: Iterators ...
An iterable is defined as an object that has an __iter__ method, which is required to return an iterator object. An iterator in...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found