Problems converting different networks.

I succesfully tried the repo on some network in the examples, however i encountered some issues trying to convert different networks. I use a jetson nano with Tensorrt 5.0.6 and torch 1.2.0 First i tried to convert Inceptionv3 from torchvision models, the network is not converted and the error is: During conversion: [TensorRT] ERROR: Unused Input: i Then during inference: AttributeError: 'NoneType' object has no attribute 'get_binding_index'

I also tried to convert an implementation of SSD-mobilenet-v2 that can be found at: “https://github.com/qfgaohao/pytorch-ssd” this time the conversion script runs without any problem however the resulting network gives completely wrong score prediction, what could be the reason?

For both the cases there was the unsupported operation torch.unsqueeze that is a simple addition of a singleton dimension, if i understood it correctly, for this reason i used the same converter of function view to implement it.

[redzhepdx]

You can create your own wrapper for the torchvision models or any model that contains a dictionary output(OrderedDict etc). Here is my solution for torchvision.detection models’ backbones:

[Tested with retinanet_resnet50_fpn]

class BackboneWrapper(nn.Module):
    def __init__(self, model: nn.Module):
        super(BackboneWrapper, self).__init__()
        self.model = model

    def forward(self, x) -> List[torch.Tensor]:
        outputs = self.model(x)
        return [output for name, output in outputs.items()]

Usage :

backbone = BackboneWrapper(retina_fpn_model.backbone)
trt_backbone = torch2trt(backbone, [x])

Also to be able to re-create the model from the saved backbone you will need another wrapper to map these output tensors into a dictionary with corresponding tags(Probably there will an expectation in the main forward function of the model).

class BackboneDeWrapper(nn.Module):
    def __init__(self, model: nn.Module, return_names: List[int]):
        super(BackboneDeWrapper, self).__init__()
        self.model = model
        self.return_names = return_names

    def forward(self, x) -> OrderedDict:
        outputs = self.model(x)
        return OrderedDict({str(name): output for output, name in zip(outputs, self.return_names)})

Usage :

backbone = torch.load("saved_backbone")
returned_layers = [2, 3, 4, 6, 7] # P2, P3, P4, P6 and P7 for fpn
model.backbone = BackboneDeWrapper(backbone, return_names=returned_layers)

[jaybdub]

It’s possible that an unsupported operation was not detected. Currently, it’s not perfect how this is done (primarily looks for functions under torch.xxx, torch.nn.xxx, torch.nn.functional.xxx.

For the SSD model in particular, there are likely operations related to anchor box parsing that are not supported. Typically though, object detection models contain a backbone CNN, which likely is supported.

I haven’t investigated the model you sent in particular, but it may look something like

backbone_trt = torch2trt(model.backbone, [data])  # may vary, for illustrative purposes

model.backbone = backbone_trt

Are you able to determine if this method matches your use case?

Best, John