[RFC][Rust][Roadmap] Expand Rust capabilities

The one thing that common to Rust programmers is that, if it’s not written in Rust, it ~probably~ almost definitely should be. In the case of TVM, we can do this rather painlessly and incrementally since the “frontend” (i.e. bindings) made by @ehsanmok allow us to replicate the functionality of c++ TVM until the pure-Rust portion (currently a CPU-only static runtime) is feature complete.

The target for the next TVM release (0.6) will be (in order):

• upgrade to Rust 2018 (#2524)
• unify bindings and runtime under a single tvm namespace
  • use single bindgen-generated ffi in common, remove tvm-sys
  • use common types for packed_func, module, (DL)Tensor
• make bindings optional using feature flag and conditional compilation
• use newer failure crate for errors
• Rust DeviceAPI (#2525)
• Relay graph runtime
• assess feasibility of porting compiler (incl. python bindings)

For those wondering “why even??,” my main response would be that rust is more ergonomic than C++ (having powerful macros, a package manager, a sane standard library); safety is also nice to have but mostly for the runtime. In an ideal world, all of the boilerplate associated with creating new passes and operators is handled by a few proc_macros and some build.rs scripts–same with generating the python bindings (complete with docs; ref: https://github.com/dmlc/tvm/pull/2328#issuecomment-450001679).

[tqchen]

Great discussions 😃 First of all, as many people said, I think the core of TVM will stay C++ in the foreseeable future.

On the other hand, the question is that, can we enable the use of rust(optionally), along with C++ core? I think the answer is yes. Let me elaborate that a bit. The TVM project is a layered system: we have runtime, frontend and compiler core. So we should really ask what does it mean to introduce rust in each setting.

Rust Runtime Backend

Runtime backend means implementing the C APIs in include/tvm/runtime/c_runtime_api.h. TVM runtime is the standalone thing that allows us to implement all the runtime features, in this case, the answer is definitely yes. The main reason is rust have good WASM support as @nhynes mentioned and probably better SGX support. Having a rust runtime gives us good options in these settings. Also because the runtime is minimum, having a rust alternative version helps us to build good support for WASM and SGX.

Thanks to @nhynes , we already have good rust runtime backend support, and I think it makes sense to think about supporting graph to some extent to get good WASM support(without libc).

Rust Frontend Support

Frontend means wrap C APIs in include/tvm/runtime/c_runtime_api.h to provide a rust abstraction. It allow us to deploy models using rust. I think it also makes sense, and @ehsanmok has been working on that.

Compiler Core and Node System

So the most interesting questions come when we want to think about Rust for compiler cores. In particular, we should ask where should the AST(i.e. TVM’s Node class) live. Nodes are important not only because they are in C++, but also because we have clean API to expose them to python. While I have no doubt it is possible to re-implement the Node system in rust, it makes C++ obsolete, because you cannot directly visit these memory object directly as struct pointer access in C++, so it is not a realistic option.

A more realistic, but still a challenging question, is whether can we enable writing passes in rust, and plug them in as PackedFunc to the C++ core. I have some serious thoughts in that direction. The simplest way is to use frontend API, like python. However, that is only good for prototyping purposes, because we need to do C API calls just like python did.

A Common Node System

What we really want in our case, is to make rust access the C++ AST as backend language, so the Node field access can be done natively via struct address accessing, without getting into C API. Because some folks in the community expressed interest in Rust, I gave some serious thought into this.

I think it can done, but also technically challenging. The general idea is to make use of C ABI. That is, we build most of the Node system as C objects, which then allows us to expose them directly to Rust. The primary AST and core are still in C++, but the rust language module can directly access these in-memory AST fields via pointer de-referencing. The key challenges here, however, is how can we solve the problem of virtual functions tables, since they are part of C++ ABI and not visible in Rust. For the base Node system, that is partially achievable by defining our own version of vtable struct. In that way, we might be able to expose some of the Nodes partially to rust. I have not tried it out, but it could be an interesting step as long as it does not change the current c++ code base.

Summary of My Take

• We should support rust runtime/frontend, and get most of the benefit @nhynes mentioned(riscv, wasm).
• We should not rewrite codebase in rust, but we could try to explore possible directions to enable rust module plugin alongside with the current c++ core.

Most of the current @nhynes 's proposal makes sense since they fall into support rust runtime, perhaps we should change the title to Enable rust support instead of rewriting 😃

[nhynes]

@FrozenGene understood. I will extract the high-level idea of your comment and say that the development of any Rust functionality will be independent of the main project. Although I jokingly suggested that the reason to rewrite in Rust is because of the language alone, doing so would have the benefits of easier extensibility/prototyping and bringing any re-implemented functionality to new platforms like riscv. For instance, imagine having an in-browser editor for TVM modules backed by a Wasm compiler enabled by Rust; or a F1 instance running a riscv soft core with VTA. As @Ravenwater mentioned, many of us (myself included) are trying to do science. Having more options as long as it’s not a distraction to the core project can only help that goal.