Wrap ES 2018 Asynchronous Iterators

I’d like to consume ES 2018 asynchronous iterators(see also the 2alities intro on the topic) as Bacon.EventStreams by wrapping the iteration by Bacon.fromBinder. This general interface would Bacon.js allow to work with new data sources. E.g. Node.js Readable Streams are Async Iterators now (Although here one could use Bacon.fromEvent('data') too.)

However, I cannot decide how to reflect the loss of subscribers in the Bacon.EventStream in the iteration of the AsyncIterable.

1.) Should the re-attachment of EventStream subscribers commence iteration at the point where the last previous subscriber was detached? I.e. should the iteration be paused? Or, 2.) should the iterator be returned, i.e. allowed to clean up, and the stream ended, when the subscriber count reaches 0?

Quick example:

var ten = countDown(10); // countDown is an async generator function 
var tenStream = pausableFromAsyncIterator(ten); // implemented using Bacon.fromBinder
tenStream.take(2).log()
// --> 9, 8, <end>
tenStream.take(2).doEnd(() => ten.return()).log()
// --> 7, 6, <end>

// Or
var tenStream = fromAsyncIterator(countDown(10)); // implemented using Bacon.fromBinder
tenStream.take(2).log()
// --> 9, 8, <end>
tenStream.take(2).log()
// --> <end>

What is the Bacon.js way to do it?

Let me formulate the question differently: Iterators are stateful (i.e. they keep the current “postition” of the iteration). Should we expose this statefulness in making the Bacon.EventStream stateful as well?

I know that for instance Bacon.fromArray and Bacon.sequentially commence publishing array items form the position where the loop position has been when the last subscriber was detached. Also reading a file every time from the beginning might not what one wants. BUT 1.) would mean that we need to share the AsyncIterator with other code parts (e.g. .doEnd(() => asyncIterator.return()) in order manage its lifetime or leaking memory. This would add complexity which I don’t like.

On the other hand 2.) might not be what the programmer familiar with Iterators would expect, BUT: Is unsubscribing and later re-subscribing to the same Bacon.EventStream really a programming pattern Bacon.js should support?

[semmel]

For now I use a rather naive recursive (see implementation below) adapter which lets me digest asynchronous iterables with Bacon.js like this:

Bacon.once(path.join(os.homedir(), 'tmp/hello.txt')).
flatMap(_ba.fromAsyncGeneratorFunction(
   path => fs.createReadStream(path, {encoding: 'utf8'})
))
.log() 
// --> Hello Word! <end> (or whatever the file contains)

Implemented so

const
isAsyncIterable = obj => Symbol.asyncIterator in Object(obj),

/**
 * @typedef BaconAdapters.fromAsyncGeneratorFunction
 * @description As fast as possible pulls values from the iterator returned by the generator function.
 * The returned EventStream is the sequence of the pulled values. When the EventStream has no more subscribers
 * the pulling is finished on the next value yielded by the iterator.
 * @function
 * @template T
 * @param {function(...*): (AsyncIterator<T> | AsyncIterable<T>)} generatorFn
 * @return {function(...*): Bacon.EventStream<T>}
 * @example
 * async function* countDown(n) {
 *    for (let i = n; i--; i > 0) {
 *       yield await new Promise(resolve => setTimeout(resolve, 1000, i));
 *    }
 * }
 * fromAsyncGeneratorFunction(countDown)(3)
 * // -> 2, 1, 0, <end>
 */
fromAsyncGeneratorFunction = generator => (...args) => Bacon.fromBinder(sink => {
   const
      iteratorOrIterable = generator(...args),
      
      /** @type {AsyncIterator} */
      iterator = isAsyncIterable(iteratorOrIterable)
         ? iteratorOrIterable[Symbol.asyncIterator]()
         : iteratorOrIterable,
      
      pullNext = () => {
      iterator.next().then(({value, done}) => {
         if (done) {
            sink(new Bacon.End());
            return;
         }
         if (sink(value) !== Bacon.noMore) {
           pullNext();
         }
      },
      error => {
         if (sink(new Bacon.Error(error)) !== Bacon.noMore) {
           pullNext();
         }
      });
   };
   
   pullNext();
   return () => { iterator.return(); };
});

[semmel]

@raimohanska No need to apologise! I just needed some feedback that my idea is not crazy or if I had missed something important.

Right now I can take care of outright bugs and fixes but anything that requires more than an hour will most likely drop.

👍 That’s excellent nevertheless!

Bacon.js is a well working reactive library without great omissions. Thanks to .fromBinder, .withStateMachine and .transform users can extend the functionality. Bug fixes is basically all what’s needed to keep my daily work going 😄