Make TorchServe multi framework

We’ve been assuming so far that Torchserve can only work with Pytorch Eager mode or Torchscripted models but our current handler is general enough to make it possible to support ONNX models.

The idea is a hack one of our partners mentioned that involves

1. Adding onnx as a dependency in docker file or requirements.txt
2. Loading onnx model in initialize handler
3. Making an inference in the inference handler

It may not necessarily be the best way to serve ONNX models but it lets people avoid having to use a different serving infrastructure for each different type of model

This is a good level 3-4 bootcamp task - the goal would be to

1. Get a Pytorch model like Resnet 18
2. Export it using ONNX exporter
3. Run and inference with it in an ONNX handler and submit it as an example in this repo

[msaroufim]

This was just merged, will be featured in next release today

[joaquincabezas]

Hello @msaroufim

Thanks for your initiative! Would love to see Torchserve serving ONNX “out-of-the-box”. Any feedback on these benchmarks?