Graph Diagrams

Add a construct_graph function to Tensors that creates a visual graph of a network.

Example:

"
 /‾‾‾\
| out |
 \___/
   Λ
   |
   |
   |
 |‾‾‾|
 | + |
 |___|
   Λ
   |
   |‾‾‾‾‾‾‾|
   |       |
   |       |
 /‾‾‾\   /‾‾‾\
|  Z  | |  K  |
 \___/   \___/
           Λ
           |
           |
           |
         |‾‾‾|
         | * |
         |___|
           Λ
           |
      |‾‾‾‾‾‾‾‾‾|
      |         |
      |         |
    /‾‾‾\    /‾‾‾\
   |  X  |  |  Y  |
    \___/    \___/
"

[davidmascharka]

Definitely a nice feature to have. A couple suggestions:

Rendering

Figuring out positions for your variables is definitely not an easy problem. You’re essentially trying to solve an entire typesetting problem. It’s very interesting, but there’s a ton of existing work that you can (and should unless you’re just super interested in typesetting algorithms) take advantage of to handle the rendering for you.

One great library is Graphviz which I personally use for typesetting graphs. Since you’re trying to draw a computational graph, a graph rendering library is probably a good place to start. It’s relatively easy to get up-to-speed with. You’d probably want to use something like pydot, which is a Python interface to the DOT language, which is super useful for directed graphs (like a computational graph).

Another advantage is it looks prettier than ASCII art (not that there’s anything wrong with ASCII art).

Creating the Graph

You can leverage the fact that you have to create a computational graph of anything you’re interested in rendering anyway, so you don’t really need to store any extra information. You can essentially just follow the path of null_gradients and use that to construct and render your graph object.

[petarmhg]

Implemented in #70