how to throttle and then watchLatest, can we compose high-level helpers?

Taking example from https://github.com/yelouafi/redux-saga/issues/70#issuecomment-182475883

function* fetchUser(action) {
   try {
      const user = yield call(Api.fetchUser,action.payload.userId);
      yield put({type: "USER_FETCH_SUCCEEDED", user: user});
   } catch (e) {
      yield put({type: "USER_FETCH_FAILED",message: e.message});
   }
}

function* mySaga() {
  yield watchLatest("USER_FETCH_REQUESTED",fetchUser);
}

Now imagine that if 2 USER_FETCH_REQUESTED gets dispatched very close to each others, instead of firing 2 requests, and then cancelling the first one, throttling would permit to avoid doing the 1st request in the first place.

I don’t have a real usecase for this but as we are going to implement high-level apis (so probably throttle?), can it be possible to compose them?

[yelouafi]

@AriaFallah

It ended up being a long response. I hope It’s not completely non-sens 😃

I tend to view the difference from a pull vs push perspective. Fundamentally it relates to how you deliver the result of a function call

push vs pull

pull

The return value of the function is delivered via assignement.

const result = func(...args)

Above We are pulling the result from func. This is the normal way we’are all used to.

push

The return value of the function is delivered in the argument position (via callback).

func(...args, arg => ...)

The result is pushed into the callback passed as an additional parameter to the function.

iteration vs Observation

Another perspective is iteration vs observation, which extends the concept of pull/push from single values to collections (or more generally from scalar to compound values).

Iteration

Happens when you are pulling values from a collection in sequence

// iterable: something that can returns an iterator
const iterator = iterable[Symbol.iterator]

// iterator: has a next() method and returns a { value, done } result
let result = iterator.next()
while(!result.done) {
  //... do something with result.value
  result = iterator.next()
}

The iterator acts as a producer of data and the while loop acts as a consumer of data. With iteration the consumer is always in control: you decide when to pull the next value

Observation

The typical example is with Rx like observables

// observable: has a forEach(onNext, onEnd) method
// Not quite the Rx signature, just simplifying for illustration
observable.forEach(
/* onNext */ val => // ... do something with val
/* onEnd */  ()  => // ... do something when ended
)

Same as above, the observable acts as a producer of data and the provided callbacks act as a consumer of data. But here the control is inversed, It’s the producer who decide when to push the next value.

When to use each one

The pull/iteration approach is more suitable when dealing with control flow. It maybe related to how human brain work, people find it easier to reason about a program written in the pull style.

It maybe also related to our legacy. Most of the accumulated Programming Knowledge (AFAIK) was in the pull side: starting from algorithms, control structures (if, while, for), routine/subroutine… So most of our learning & experience is also done on the pull side.

But one thing is sure, we’re always trying to transpose problems into the pull side (async/await, Generators)

Another advantage has to do with composability. If your function returns a value, it automatically makes it composable with other functions

// only possible if f and g both return something
result = f(g())

// same as; 
x = g()
y = f(x)

With a pure push style (like in Node API) this is not possible, because callback style functions return undefined: and you can’t do anything useful with undefined. You can learn more about in an old post I wrote From Callback to Future -> Functor -> Monad (do not feel intimidated by the terms in the title, the article is plain familiar English 😃 )

OTOH The issue with a pure pull approach is when we deal with future values. I mean, when we call a function that may take some noticeable time before it can delivers its result (a common case is when you make a call to a remote server)

// server data may take a long time to return
result = getServerData()

// doSomething with result

The problem is how to deal, from the execution POV, with the getServerData() call ? Should we block the program until the function return ? clearly, this is not a good idea, because the program have certainly other things to do while waiting for the server result.

Typically, those other things may involve responding to User interactions if We’re in a UI program, or responding to network requests if we’re in a server environment.

For a long time, the traditional way to deal with this issue was by using Threads, which are like separate programs running in parallel and typically managed by the Operating system. For example in a language like Java with a thread based API, if you do

// will suspend the current thread until the server returns
result = getServerData()

doSomething(result)

This is a pull style approach, the runtime manages transparently the suspend/resume mechanics, so the developer can focus on writing code using a familiar synchronous style. As you see, pull style was there long before async/await but …

The issue with the above approach is thread usage. Thread have a non negligible cost in terms of memory/performance. So to make it short, here come push based solutions like Node with its callback model.

getServerData( result=> doSomething(result) )

// continue and handle other things

So, no threads, no suspension, just one thread to rule them all.getServerData will tell the runtime: I’m going to continue my business. When a result come, call this callback, it will know what to do.

Edit: And anyway, JavaScript is single thread; So we can’t implement a thread based solution on it even if we’d like to do it

The main advantage of this approach is performance, since there is no memory/performance overhead of threads usage.

But this push approach isn’t only suitable for technical reasons. It may become necessary when we deal with events. here, it’s more a conceptual constraint: we want to do some work when some event happens. The decisive moment here is when the event happens, so it makes plain sens that the event producer takes control in this case: ie a push approach is more suitable.

Are Promises/Observable a push concept ?

Not exactly, the Promise/Observable concepts advantage is to wrap the callback push concept in an abstraction (e.g. a Promise is like a magical box holding the eventual content of a Future result). Then you can use this abstraction and pass it around to your program as a first class value (i.e. use either as argument or return value).

So now with promises we have the same Java/thread based code, except one thing: instead of returning a plain value we return a Promise

// result is a promise
result = getServerData()

result.then(value => doSomething(value) )

Note also the whole result.then(value => doSomething(value) ) will result in another Promise if you return some value from doSomething. So you can also continue in a sequential/pseudo-synchronous style

Observable, extends the Promise abstraction further to deal also with event observation

// result is an Observable
result = domElement.on('click')

result.forEach(eventData => doSomething(eventData) )

So There is a pull because now the we can get something from the functions. And this is very important: if a function returns something, it means the function can be composed with other functions; If a function returns undefined it means you hit a wall. With composition you never hit a wall.

But there is still a push because result.then/results.forEach deliver their results in the argument position. So it’s more an interleaving of push/pull (And it’s always like that in other solutions, just matter of what the library exposes and what it hides from you)

So for the Promise case, come async/await

async function myfunc() {
  // result is a plain value
  result = await getServerData()

  doSomething(result)
}

We get the same synchronous style as in the Thread based solutions but without the costs incurred by thread usage.

Now the next step is to extend the above sync model to Observable. Here enter Generators

Generators as iterators and observers

If Generators are capable of replicating the async/await model. it’s because they encapsulate both the iterator and the observer pattern

Generators as iterators

The mostly-known usage of Generators

// produces an infinite sequence of 0, 1, 2 ...
function* naturalNumbers() {
 let k = 0
 while(true) {
   yield k
   k++
 }
}

const iterator = naturalNumbers()

iterator.next().value // => 0
iterator.next().value // => 1
iterator.next().value // => 2

Generators as observers

This is a less known side of Generators

function* logEvents() {
  // wait for a value that will get passed as argument to next
  while(true) {
    const event = yield;
    console.log(event)
  }
}

const observer = logEvents()
observer.next() // forward one step since w're not interested on yielded values
document.addEventListener('click', observer.next.bind(observer))

The Generator doesn’t produce any value, it just sits and waits for something to get passed to it. The Generator driver here is in control: it can advance the Generator by providing it an input.

But note the Generator doesn’t produce observables, it just produce observers; there are some possible ways to achieve this: You can read more about one proposed solution here (https://github.com/jhusain/asyncgenerator)

What about redux-saga

TBH, I can’t even pretend there was a conceptual phase where the model was specified before implementation. I started it as a Proof of Concept following a discussion where @slorber introduced me to the concept of Saga (in the case of React/Redux it acts like a mediator between Components and Reducers).

To describe the model a posteriori. I could say it’s a more imperative model than FP (like Observables/Promises). Each Saga can be connected to an input/output and can fork/join/cancel one or multiple tasks.

Another difference is that it’s a kind of Sand boxed model: everything happens inside the Generator by yielding effects.

If you’d like to compare it to Observables, you may see it as a super Observer which can handle/coordinate input from different Observables.

Edit

Also We can also say that redux-saga introduces a pull model of events. where in observables you’d write

eventObservable = someSource.on('click')
eventObservable.forEach( event => doSomething(event) )

With redux-saga it translates to

// assuming saga is connected to someSource
function* saga() {
  while(true) {
     const event = take('click')
     doSomething(event)
  }
}

[gaearon]

OTOH let’s not forget we want the library to be easy to use without having a good command of FP style.