[RFC] [Relay]A Normal Form Conversion

https://github.com/dmlc/tvm/pull/2251 implement a pass which converts expression with sharing (different occurrence of Expr might be the same ptr) into expression without sharing, by inserting let all over the place.

It is useful because it allows subsequent passes to ignore sharing and only deal with let (which might also reside in the original program anyway).

However, to do so, we must define precisely what is the scope of a node. It is determined by the Expr that immediately depends on it. For example in the Graph a + b, a + b immediate depends on both a and b. We will call ‘immediately depends on’ ‘point to’ from now on.

Let’s start from the degenerate case that there is only one graph a that points to b. In that case, GetScope(a, b) = scope(a) if (b is not in curly brace position of a) else new_subscope(scope(a))

Now the other case. If nothing points to node b: b is the outmost scope

If there are multiple pointer pointing to b: b’s scope is the least common ancestor of all GetScope.

[MarisaKirisame]

@junrushao1994 0: we decided to used anf instead of cps because the latter is more complex (not just the transformation, but subsequent optimization is harder to write in cps style). also we do not need continuation. 1: the complexity is somewhere around n log n, so it would not be an issue. also other passes will have a higher complexity, so this is not a bottleneck afaik. also even if the nn is 10k layers we might simply represent it as a compact program.

[junrushao]

@tqchen in your example, %1 = %x -1 should be %1 = %x, which is a typo