[RFC][GOLANG] TVM Golang Runtime Interface Discussion

Here briefing the golang interface design aspects.

Few golang concepts

• Usage of packages C and unsafe golang packages to go across golang boundary while accessing functions, types and pointer …etc.
• Slices: Usage if golang slice where ever is possible to simplify application programming. Example being use slice of string instead of argument pair containing argc, argv

Please refer to golang tutorials for details of cgo, unsafe, slices …etc.

Objective

• Initial version is expected to cover basic inference API listed below.
• Similar to many programming languages a package which can be imported and used is the best option here too.
• Hence the objective of this effort is a golang package gotvm which is wrapped over TVM c_runtime_api.

API

This initial version expected to cover the below API

func FuncListGlobalNames() (retVal []string, err error)
func GetGlobalFunction(funcname string) (retVal func(args ...interface{}) (*Value, error), err error)
func RegisterFunction(args ...interface{}) (err error)
type Array
    func Empty(shape []int64, args ...interface{}) (parray *Array, err error)
    func (parray Array) AsSlice() (retVal interface{}, err error)
    func (parray Array) CopyFrom(val interface{}) (err error)
    func (parray Array) GetCtx() (retVal Context)
    func (parray Array) GetDType() (retVal string)
    func (parray Array) GetNdim() (retVal int32)
    func (parray Array) GetShape() (retVal []int64)
type ByteArray
    func NewByteArray(val []uint8) (retVal *ByteArray)
    func (tbytearray ByteArray) GetData() (retVal string)
type Context
    func CPU(index int32) Context
    func CPUPinned(index int32) Context
    func GPU(index int32) Context
    func Metal(index int32) Context
    func OpenCL(index int32) Context
    func OpenGL(index int32) Context
    func ROCM(index int32) Context
    func SDAccel(index int32) Context
    func VPI(index int32) Context
    func Vulkan(index int32) Context
type Function
    func ConvertFunction(args ...interface{}) (fhandle Function, err error)
    func (tvmfunction Function) Invoke(args ...interface{}) (retVal *Value, err error)
type Module
    func LoadModuleFromFile(modpath string, args ...interface{}) (retVal *Module, err error)
    func (tvmmodule *Module) GetFunction(funcname string, args ...interface{}) (retVal func(args ...interface{}) (*Value, error), err error)
type Value
    func (tvmval Value) AsFloat64() (retVal float64)
    func (tvmval Value) AsFunction() (retVal *Function)
    func (tvmval Value) AsInt64() (retVal int64)
    func (tvmval Value) AsModule() (retVal *Module)
    func (tvmval Value) AsStr() (retVal string)

Sample Code

/*!
 *  Copyright (c) 2018 by Contributors
 * \brief Sample golang application deployment over tvm.
 * \file simple.go
 */

package main

import (
    "fmt"
    "runtime"
    "./gotvm"
)

// NNVM compiled model paths.
const (
    modLib    = "./deploy.so"
)

// main
func main() {
    // Welcome
    defer runtime.GC()
    fmt.Printf("TVM Go Interface : v%v\n", gotvm.GoTVMVersion)
    fmt.Printf("TVM Version   : v%v\n", gotvm.TVMVersion)
    fmt.Printf("DLPACK Version: v%v\n\n", gotvm.DLPackVersion)

    // Import tvm module (so)
    modp, _ := gotvm.LoadModuleFromFile(modLib)
    fmt.Printf("Module Imported\n")


    // Allocate Array for inputs and outputs.

    // Allocation by explicit type and context.
    tshapeIn  := []int64{4}
    inX, _ := gotvm.Empty(tshapeIn, "float32", gotvm.CPU(0))

    // Default allocation on CPU
    inY, _ := gotvm.Empty(tshapeIn, "float32")

    // Default allocation to type "float32" and on CPU
    out, _ := gotvm.Empty(tshapeIn)

    fmt.Printf("Input and Output Arrays allocated\n")

    // Fill Input Data : inX , inY
    inXSlice := []float32 {1, 2, 3, 4}
    inYSlice := []float32 {5, 6, 7, 8}

    // Copy the data on target memory through runtime CopyFrom api.
    inX.CopyFrom(inXSlice)
    inY.CopyFrom(inYSlice)

    fmt.Printf("X: %v\n", inXSlice)
    fmt.Printf("Y: %v\n", inYSlice)

    // Get function "myadd"
    funp, _ := modp.GetFunction("myadd")

    // Call function
    funp(inX, inY, out)

    fmt.Printf("Module function myadd executed\n")

    // Get the output tensor as an interface holding a slice through runtime CopyTo api.
    outSlice, _ := out.AsSlice()

    // Print results
    fmt.Printf("Result:%v\n", outSlice.([]float32))
}

@dmlc/tvm-team welcome to comment on this info.

Please refer to #1470 to have a look at initial effort.

[tqchen]

One possible problem of the current callback interface is that the value types seem to be pretty strict, for example, we do pfunc.Invoke((int64)(0)), but ideally, we want to support pfunc.Invoke(0).

One option is to Value as a signature for the implementation of callback could be helpful. So callback’s signature takes in Value and call AsXXX during to get the argument and construct Value during return.

The invoke expose the signature (args ...interface{}) Value, and we allow user to directly pass in int/closure/Value/float/Array etc and do the automated conversion to Value under the hood

[tqchen]

OK, I think it is fine to use the interface if go users are familiar with that. One second thought on the callbacks, we do need a way to convert functions into handles and pass PackedFunc handle back when there is a function argument. We can do it either by attaching a field to the closure(if go permit that, as in https://github.com/dmlc/tvm/blob/master/web/tvm_runtime.js#L466), or we do need a customized Function struct, and use a special method Call to call the function