all lazy structures are not stack safe

All lazy statures (IO, Future, State) have stack issue. general way of implementing operations on such structures is:

function Structure(f) {
  if (!(this instanceof Structure)) {
    return new Structure(f);
  }
  this.run = f
}

Structure.prototype.method = function (a){
  return new Structure(function(){
    // here we call `run` from a or something like that 
  })
}

All of lazy structures need extra stack frame on each map/ap/chain/..., as they are creating new functions which have use old computation or passed in function, requiring larger stack size, leading to stack overflow (when you run/execute/lower them)

1. apply curried function f which is taking 10000 arguments:

...(T.of(9999).ap(T.of(10000).ap(T.of(f)))

1. map on some structure 10000 times

T.of(1).map(id).map(id).map(id).map(id).map(id).....

btw there is no such issue with native Promise:

Array(40000)
  .fill(idx => a=>a + idx)
  .reduce(
    (p,f,idx) => p.then(f(idx)), 
    new Promise((res,rej) => { setTimeout(res, 1000, 1)})
  ).then(console.log.bind(console))// 799980001

General way to fix this would be rewrite such structures so that they builds up computation as data on map/ap/chain/...(like Free monad/applicative…) and executing run interpret it in a stack safe way. For example this way we can make function composition safe. Also related paper “Stack safety for free” by @paf31.

what others think on this issue?

[robotlolita]

@SimonRichardson PTC is still in the spec, and VM writers are supposed to implement it. There’s a proposal to require a token for calls that should be a tail call, but that hasn’t left draft stage yet, and as I understand part of TC39 objects to that (in particular because you don’t really have PTC at that point) — either way, we’d get some sort of tail calls though.

Tail calls solve the same problem trampolining does, they’re just cheaper (you can translate them to a simple jump with the appropriate registers / argument stack updated), you just need to write your code such that all relevant calls would be in tail position (not the case with the Io example above — f(this.run()) doesn’t have this.run() in tail position —, but Data.Task’s current implementation pretty much translates things into CPS, as does fantasy-promises in this organisation, since you need to for asynchronous code anyway).

So:

var Io = (fork) => ({
  run: fork,
  of: (value) => Io(f => f(value)),
  chain: (f) => Io(g => fork(x => f(x).run(g))),
  map: (f) => Io(g => fork(x => g(f(x))))
});

// This:
io(f =>  f(1)).map(x => x + 1).map(x => x * 2).run(x => x);

// Translates into:
(h =>
  (g =>
    (f => f(1))
      (x => g(x + 1)) // first map
  )(y => h(y * 2)) // second map
)(x => x)

Because all of these calls are in tail position, the VM doesn’t need to return a value to once it finishes evaluating the function, so it can translate it to something like:

JUMP a

a: -- f(1)
PUSH 1
JUMP b

b: -- g(x + 1)
PUSH 1
ADD
JUMP c

c: -- h(y * 2)
PUSH 2
MULTIPLY
JUMP d

d:  -- x
RET

Doing CPS translations manually to verify these is quite a pain, but you really only need to ensure that your call happens in tail position. If it does, the VM doesn’t need to translate that into a CALL instruction, it can just use a JUMP, and then you don’t have a stack problem.

The only problem with this so far is that not all VMs have implemented PTC 😦

[Avaq]

It may be worth noting that Fluture hasn’t been affected since its 6.x release. Since that means that providing a stack safe implementation can be up to the library author, I suppose this issue could be closed.