Document upstream gradient behavior for functions with multiple outputs

When a function has several outputs and the user backwards through one of them, it is not obvious that all of the other upstream gradients are collected. This is probably most true when the last function in the graph has multiple outputs. The following is such a case.

import numpy as np
import chainer
from chainer import Variable

x = chainer.Variable(np.arange(4, dtype='f'))
ys = chainer.functions.split_axis(x, 2, axis=0)
for y in ys:
    y.grad_var = chainer.Variable(np.full_like(y, 3, dtype='f'))
ys[0].backward()
x.grad  # [3, 3, 3, 3]. Some users might expect [3, 3, 0, 0] since ys[1] is not involved?

How about document this behavior, or is it already?

[beam2d]

This feature has been used through a trick when one wants to start backprop from multiple root variables with Variable.backward. When one wants to start backprop from variables x and y, one can feed them to F.identity and then call backward on one of the outputs.

x, y = F.identity(x, y)
x.grad = ...
y.grad = ...
x.backward()

I know that this snippet is too tricky, and it’s not good to let users rely on such a trick, but we at least should provide an alternative way to accomplish the same goal if we remove the feature discussed in this issue. One idea is to provide a functional version of Variable.backward, say chainer.backward(ys), which accepts multiple Variables to start with.

[stale[bot]]

This issue is closed as announced. Feel free to re-open it if needed.