[Question] C++ Inference

Question

Hello, I am using SB3 to train some model where I want the inference to run on embedded robots using C++. I had a look at PyTorch documentation and doing this is not very hard “by hand”, but the process could indeed be automated.

I could maybe contribute, but I would like some questions to be answered about how do you imagine the feature.

In my opinion, there is more that can be done that simply documenting it, because we could help to automate the observation and action rescaling and normalizing, in a way such that methods like model.predict gets converted seamlessly to C++.

Here is how I would see it:

• We provide some piece of C++
• We provide the methods that export the JIT PyTorch models (.pt) from the different agents implemented in SB3
• We provide some code that generates some C++ (at least we need to be able to export the action and observation definitions so that the pre/post processing of observations and actions are implemented in C++ as well)

What do you think ?

Additional context

Checklist

• I have read the documentation (required)
• I have checked that there is no similar issue in the repo (required)

[Gregwar]

I started working on that in the RL Zoo, I will indeed open a draft PR soon, even if it will not support everything it can be used as a base for discussions

I have a very short term goals of embedding inferences in our humanoid robots so I will also be the first user

[Gregwar]

My list is a list of features.

About questions, I mostly wanted to know what was the recommended direction for this. Making it some contrib to RL Zoo indeed looks like the way to go, since the export can be achieve “from the outside” or SB3.