App crashes with memory leak when running for a long time

Version

0.2.2

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Following the steps below, the memory heap increases at a pace of about 100MB per hour. We are developing an application that needs to run for long periods of time. Could you please tell me how to avoid this issue.

• Create empty project

$ npx react-native init AwesomeTSProject --template react-native-template-typescript
$ cd AwesomeTSProject

• Add PlayTorch to Existing App

$ yarn add react-native-pytorch-core
$ npx pod-install

• Rewrite App.ts based on Playground

import * as React from 'react';
import {useCallback} from 'react';
import {LayoutRectangle, StyleSheet} from 'react-native';
import {
  Camera,
  CameraFacing,
  Canvas,
  CanvasRenderingContext2D,
  Image,
  ImageUtil,
} from 'react-native-pytorch-core';

const App = () => {
  const contextRef = React.useRef<CanvasRenderingContext2D>();
  const [layout, setLayout] = React.useState<LayoutRectangle>();

  const handleCapture = useCallback(
    async (image: Image) => {
      const context = contextRef.current;
      if (context != null && layout != null) {
        context.clear();
        const imageWidth = image.getWidth();
        const imageHeight = image.getHeight();
        const scale = Math.min(
          layout.width / imageWidth,
          layout.height / imageHeight,
        );
        context.drawImage(image, 0, 0, imageWidth * scale, imageHeight * scale);
        const wholeImageData = await context.getImageData(
          0,
          0,
          imageWidth * scale,
          imageHeight * scale,
        );
        const wholeImage = await ImageUtil.fromImageData(wholeImageData);
        await wholeImageData.release();
        await wholeImage.release();
        await context.invalidate();
      }
      image.release();
    },
    [contextRef, layout],
  );

  return (
    <>
      <Camera
        onFrame={handleCapture}
        hideCaptureButton={true}
        style={styles.camera}
        facing={CameraFacing.BACK}
      />
      <Canvas
        style={styles.canvas}
        onContext2D={context => {
          contextRef.current = context;
        }}
        onLayout={event => {
          setLayout(event.nativeEvent.layout);
        }}
      />
    </>
  );
};

const styles = StyleSheet.create({
  camera: {
    display: 'none',
  },
  canvas: {
    flex: 1,
  },
});
export default App;

However, the code below is necessary for our application and cannot be removed.

        const wholeImage = await ImageUtil.fromImageData(wholeImageData);
        await wholeImageData.release();
        await wholeImage.release();
        await context.invalidate();

• Rewrite Info.plist for camera and microphone permissions

$ vim ./ios/AwesomeTSProject/Info.plist

<key>NSCameraUsageDescription</key>
<string>$(PRODUCT_NAME) needs access to your Camera.</string>
<key>NSLocationWhenInUseUsageDescription</key>
<string></string>
<key>NSMicrophoneUsageDescription</key>
<string>$(PRODUCT_NAME) needs access to your Microphone.</string>

• Running On Device

$ npx react-native start
$ open /Applications/Xcode.app ./ios/AwesomeTSProject.xcworkspace

Expected Results

No response

Code example, screenshot, or link to repository

Related to this issue

[nh9k]

@raedle, thank you for your kindness! I haven’t profiled my app yet. My app also crashes, so I need a memory check. I’ll report back!

[raedle]

@nh9k, each image has to be released from memory manually. For context, the current implementation holds a reference in the JSContext:

• Android JSContext.java
• iOS JSContext.swift

Not releasing the image will keep the reference around.

Tensors are implemented differently. They use jsi::HostObject from the JSI API. This will eventually clean up references when the JavaScript Runtime GC finds unused references (see TensorHostObject.h).

Have you profiled your app to check what leaks memory?