Backward compatibility of ptl with live.spec on iOS

Version

0.2.0

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

1. make ptl file with live.spec.json
2. build a project with ptl on iOS
3. inference a input image
4. get this error

code: EUNSPECIFIED, 
domain: react_native_pytorch_core.BaseIValuePackerError, 
message: decodeObjectsError, 
name: Error, 
nativeStackIOS: [Array], 
stack: 
Error: decodeObjectsError

the project works well in 0.1.3, but the error comes up after upgrade to 0.2.0. Is there no backward compatibility for ptl with live.spec?

No problem on Android.

Expected Results

No error

Code example, screenshot, or link to repository

No response

[SomaKishimoto]

@raedle , thanks for resolving the issue! I’ve verified that your fix works fine.

I will try to upgrade to the new PlayTorch SDK JavaScript API by referring to your explanation and let you know if I get stuck.

Thank you always!

[raedle]

@SomaKishimoto, we found the culprit. After upgrading the LibTorch-Lite library from version 1.10.0 to 1.12.0, the kind support changed to c10::DynamicType. Unfortunately, the unit tests didn’t catch this. We have a fix in #99. When we merged the change, it will be available in the react-native-pytorch-core@nightly or the next planned release 0.2.1.

Alternatively, as mentioned, it might be worth upgrading to the new PlayTorch SDK JavaScript API. The useObjectDetection.ts in the repo should be compatible with your model’s interface (according to the live spec).

You’d have to change the mean and std for the normalize function in the packFn (see code).

// Normalize the tensor image with mean and standard deviation
const normalize = T.normalize([0.0, 0.0, 0.0], [0.0039216, 0.0039216, 0.0039216]);
tensor = normalize(tensor);

The scaleWidth and scaleHeight can be set in the resize function in the packFn (see code)

// Resize the image tensor to 3 x min(height, width)
const resize = T.resize([scaleHeight, scaleWidth]);
tensor = resize(tensor);

In the unpackFn, use the probabilityThreshold and resolve the classes (see code)

Let me know if you have question or need advice!