Improve tail call optimization documentation

Coconut: Version 1.3.1 [Dead Parrot] running on Python 3.6.3

Consider factorial written in a tail recursive form

def factorial(n, acc=1):
    if n == 1:
        return acc
    else:
        return factorial(n-1, acc*n)

When --no-tco flag is disabled this compiles to:

@_coconut_tco
def factorial(n, acc=1):
    def _coconut_mock_func(n, acc=1): return n, acc
    while True:
        if n == 1:
            return acc
        else:
            if factorial is _coconut_recursive_func_0:
                n, acc = _coconut_mock_func(n - 1, acc * n)
                continue
            else:
                return _coconut_tail_call(factorial, n - 1, acc * n)

        return None
_coconut_recursive_func_0 = factorial

When the --no-tco flag is enabled this compiles to:

def factorial(n, acc=1):
    def _coconut_mock_func(n, acc=1): return n, acc
    while True:
        if n == 1:
            return acc
        else:
            if factorial is _coconut_recursive_func_0:
                n, acc = _coconut_mock_func(n - 1, acc * n)
                continue
            else:
                return factorial(n - 1, acc * n)

        return None
_coconut_recursive_func_0 = factorial

Aren’t both the versions tail call optimized? The only difference is that the first version is optimized for all tails calls including mutual recursion whereas the second case only handles tail recursion elimination. Is that correct? I just felt that the documentation for this was not very clear since tail recursion elimination happens even with --no-tco.

---

My original question is: Is there a reason the following does not undergo tail recursion elimination on compilation?

def rec_dict(init_dict, n):
    if n == 1:
        return init_dict()
    else:
        return rec_dict(-> defaultdict(init_dict), n - 1)

where init_dict = () -> defaultdict(set)

[amacfie]

@evhub when using the kernel

[evhub]

@dileep-kishore The --no-tco flag disables tail call optimization, but not tail recursion elimination, as you’ve noticed. The reason for this is that Coconut’s tail recursion elimination is (usually) faster than doing nothing, while Coconut’s tail call optimization is (usually) slower, thus --no-tco keeps the one that leads to faster code and removes the one that leads to slower code. Mutual recursion falls into the tail call optimization not the tail recursion elimination category, thus it doesn’t get optimized if --no-tco is passed. I should definitely make the docs more clear on this.

As for your specific example, the fact that Coconut doesn’t do tail recursion elimination there is because of the lambda. Unfortunately, the way Python handles lambdas is that they store a pointer to (not a copy of!) the namespace enclosing them. Thus, if Coconut tries to re-use the namespace that produced that lambda to do tail recursion elimination, the lambda gets retroactively changed! For example, take a look at this:

x = 1
lam = lambda: x
print(lam())  # 1
x = 2
print(lam())  # 2 (!)

This is really unfortunate for doing functional programming in Python, and I haven’t yet thought of a good way to fix it other than just disabling tail recursion elimination if Coconut sees a lambda. Currently, this is undocumented behavior, so at the very least I will definitely add something on this to the docs.