Branching a Stream

What do you think about a Stream.branch function? One that would split a stream into two streams by applying a filter function. It could return a stream of two streams with shared backpressure: one with the filtered values and the other one with the rejected values. It could take two optional transform functions, which will applied as through streams to the result streams.

To explain what I mean a naive polyfill implementation:

var stream = _([0,1,2,3,4,5]);

// Branch a stream into two streams by applying a filter.
// Optionally apply through streams to the result streams.
// Returns a Stream of two streams: one with the filtered and
// another one with the rejected values.
stream.branch = function(filter, filteredTransform, rejectedTransform) {
  var filtered = this.fork().filter(filter);
  var rejected = this.fork().reject(filter);

  if (filteredTransform) {
    filtered = filtered.through(filteredTransform);
  }
  if (rejectedTransform) {
    rejected = rejected.through(rejectedTransform);
  }

  return _([filtered, rejected]);
};

You can then use it like this:

// filter
function isEven(n) {
  return n % 2 === 0;
}
// filtered stream transform
function ft(source) {
  return source
    .map(function(n) {
      return '' + n + ' (even)';
    });
};

// branch and merge
stream.branch(isEven, ft).merge().each(_.log);

Please excuse me if this is a dump question. I repeatedly found myself writing such a pattern when I wanted to take a stream and apply some calculations on a subset of values but want to retain both the computed and untouched values in the result stream.

Do you think this is beneficial at all? Should the function return a single stream, already merged both streams? Should it work with multiple branches? Is the name too general? What about tee?

[caolan]

The idea of partitioning a stream like this seems useful. Thought, I’m not sure about the API (passing in transform functions). I’d prefer the normal chaining style for transformations.

Also, I’m not sure we should assume that back-pressure should be shared here. Currently you can achieve this by using .fork() followed by .filter() (with the predicate inverted in one case). That would mean the test is run twice, however. It would be nice if we had a way to return two results in JS (without using an array).

As for naming, I prefer partition to branch. But open to other suggestions. So in general, I’m +1 on the idea of providing a partition function, but it needs some more thought.

[loveencounterflow]

@greelgorke i’m afraid if you do it this way:

var h = require('highland');
h.plugin( ... );

we’re in for trouble as any other module could have required highland and maybe obtain the same identical object h or another, very similar / deeply-equal object, meaning that it becomes a hard problem to decide which other code in the same process is affected by any changes of this h object; this may change from machine to machine and ever after doing an npm update or somesuch. prove me wrong, but i think

var h = require('highland');
h = h.plugin( ... );

is much safer.