MLKitBarcodeScanner OutOfMemoryError after +- 50 Scans

Hello,

We are using MLKitBarcodeScanner for scanning QR codes with NativeScript Angular v6.5.1. We managed to scan ± 50 QR in a row with 2-3 secondes of pause between them, and then the app freeze and we get this error:

System.err: java.lang.OutOfMemoryError: Failed to allocate a 1048592 byte allocation with 953272 free bytes and 930KB until OOM, max allowed footprint 536870912, growth limit 536870912 System.err: at java.util.Arrays.copyOf(Arrays.java:3164) System.err: at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118) System.err: at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) System.err: at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) System.err: at android.graphics.YuvImage.nativeCompressToJpeg(Native Method) System.err: at android.graphics.YuvImage.compressToJpeg(YuvImage.java:141) System.err: at com.google.android.gms.internal.firebase_ml.zzqo.zza(Unknown Source:10) System.err: at com.google.firebase.ml.vision.common.FirebaseVisionImage.zzai(Unknown Source:101) System.err: at com.google.firebase.ml.vision.common.FirebaseVisionImage.zznl(Unknown Source:51) System.err: at com.google.firebase.ml.vision.common.FirebaseVisionImage.getBitmap(Unknown Source:46) System.err: at com.tns.Runtime.callJSMethodNative(Native Method) System.err: at com.tns.Runtime.dispatchCallJSMethodNative(Runtime.java:1286) System.err: at com.tns.Runtime.callJSMethodImpl(Runtime.java:1173) System.err: at com.tns.Runtime.callJSMethod(Runtime.java:1160) System.err: at com.tns.Runtime.callJSMethod(Runtime.java:1138) System.err: at com.tns.Runtime.callJSMethod(Runtime.java:1134) System.err: at com.tns.gen.com.google.android.gms.tasks.OnSuccessListener.onSuccess(OnSuccessListener.java:19) System.err: at com.google.android.gms.tasks.zzn.run(Unknown Source:4) System.err: at android.os.Handler.handleCallback(Handler.java:873) System.err: at android.os.Handler.dispatchMessage(Handler.java:99) System.err: at android.os.Looper.loop(Looper.java:193) System.err: at android.app.ActivityThread.main(ActivityThread.java:6898) System.err: at java.lang.reflect.Method.invoke(Native Method) System.err: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:537) System.err: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858) JS: topmost() is deprecated. Use Frame.topmost() instead. JS: sizePair.pictureSize: 4608x3456 JS: sizePair.previewSize: 1440x1080 JS: Error in Firebase MLKit’s runCamera function: Error: java.lang.OutOfMemoryError: Failed to allocate a 2332816 byte allocation with 1427120 free bytes and 1393KB until OOM, max allowed footprint 536870912, growth limit 536870912 JS: topmost() is deprecated. Use Frame.topmost() instead. JS: sizePair.pictureSize: 4608x3456 JS: sizePair.previewSize: 1440x1080 JS: Error in Firebase MLKit’s runCamera function: Error: java.lang.OutOfMemoryError: Failed to allocate a 2332816 byte allocation with 745968 free bytes and 728KB until OOM, max allowed footprint 536870912, growth limit 536870912 JS: sizePair.pictureSize: 4608x3456 JS: sizePair.previewSize: 1440x1080 JS: Error in Firebase MLKit’s runCamera function: Error: java.lang.OutOfMemoryError: Failed to allocate a 2332816 byte allocation with 691768 free bytes and 675KB until OOM, max allowed footprint 536870912, growth limit 536870912

---

We’ve used the pause attribute to prevent scanning without a break (without using pause - the app crashes after approximately 8 scans in a row). We got this error when using different kinds of android devices: android 10 with 4GB RAM, android 9 with 8GB RAM, Android 6 with 4GB RAM. We also have this issue on iOS Iphone 7 (2 GB RAM).

We’ve added

android:largeHeap=“true”

in the manifest, and :

dexOptions {
    javaMaxHeapSize "4g"
  }

in app.gradle

More Information: package.json:

{
  "nativescript": {
    "id": "org.nativescript.MYID",
    "tns-ios": {
      "version": "6.2.0"
    },
    "tns-android": {
      "version": "6.5.3"
    }
  },
  "description": "NativeScript Application",
  "license": "SEE LICENSE IN <your-license-filename>",
  "repository": "<fill-your-repository-here>",
  "dependencies": {
    "@angular/animations": "~8.2.0",
    "@angular/common": "~8.2.0",
    "@angular/compiler": "~8.2.0",
    "@angular/core": "~8.2.0",
    "@angular/forms": "~8.2.0",
    "@angular/http": "8.0.0-beta.10",
    "@angular/platform-browser": "~8.2.0",
    "@angular/platform-browser-dynamic": "~8.2.0",
    "@angular/router": "~8.2.0",
    "@ngx-translate/core": "^11.0.1",
    "@ngx-translate/http-loader": "^4.0.0",
    "@nstudio/nativescript-loading-indicator": "^1.0.0",
    "nativescript-angular": "^8.20.4",
    "nativescript-background-http": "^3.4.0",
    "nativescript-camera": "^4.4.0",
    "nativescript-datetimepicker": "^1.2.2",
    "nativescript-imagecropper": "^3.0.0",
    "nativescript-imagepicker": "^7.1.0",
    "nativescript-nfc": "4.0.1",
    "nativescript-permissions": "^1.3.8",
    "nativescript-plugin-firebase": "9.0.2",
    "nativescript-theme-core": "~1.0.4",
    "nativescript-ui-chart": "^7.1.1",
    "nativescript-ui-sidedrawer": "^8.0.1",
    "reflect-metadata": "~0.1.10",
    "rxjs": "^6.4.0",
    "tns-core-modules": "^6.3.2",
    "zone.js": "^0.9.1"
  },
  "devDependencies": {
    "@angular/compiler-cli": "~8.2.0",
    "@nativescript/schematics": "~0.5.0",
    "@ngtools/webpack": "~8.2.0",
    "@types/jasmine": "^3.5.11",
    "nativescript-dev-webpack": "^1.4.1",
    "tns-platform-declarations": "6.0.1",
    "typescript": "~3.5.3"
  },
  "readme": "NativeScript Application"
}

AndroidManifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
	package="__PACKAGE__"
	android:versionCode="20003"
	android:versionName="2.03">

	<supports-screens
		android:smallScreens="true"
		android:normalScreens="true"
		android:largeScreens="true"
		android:xlargeScreens="true"/>

	<uses-sdk
		android:minSdkVersion="17"
		android:targetSdkVersion="__APILEVEL__"/>
		
	<uses-permission android:name="android.permission.CAMERA"/>
	<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
	<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
	<uses-permission android:name="android.permission.INTERNET"/>
	
	
	<uses-feature android:name="android.hardware.camera" android:required="false" />
	<uses-feature android:name="android.hardware.camera.autofocus" android:required="false" />
	<meta-data
	android:name="com.google.firebase.ml.vision.DEPENDENCIES"
	android:value="barcode"/>

	<application
		android:name="com.tns.NativeScriptApplication"
		android:allowBackup="true"
		android:icon="@drawable/icon"
		android:label="@string/app_name"
		android:requestLegacyExternalStorage="true"
		android:largeHeap="true"
		android:theme="@style/AppTheme">


		<activity
			android:name="com.tns.NativeScriptActivity" 
			android:label="@string/title_activity_kimera"
			android:configChanges="keyboardHidden|orientation|screenSize" 
 		    android:theme="@style/LaunchScreenTheme"
			android:screenOrientation="portrait">

 			<meta-data android:name="SET_THEME_ON_LAUNCH" android:resource="@style/AppTheme" /> 

			<intent-filter> 
 			<action android:name="android.intent.action.MAIN" /> 
 			<category android:name="android.intent.category.LAUNCHER" /> 
 			</intent-filter> 
		</activity>
		<activity android:name="com.tns.ErrorReportActivity"/>
	</application>
</manifest>

app.gradle:

android {  
  defaultConfig {  
    multiDexEnabled true
    generatedDensities = []
    ndk {
       abiFilters.clear()
       abiFilters.addAll(['armeabi-v7a','arm64-v8a'])
   }
  }  
  aaptOptions {  
    additionalParameters "--no-version-vectors"  
  }  
  dexOptions {
    javaMaxHeapSize "4g"
  }
} 

This is a big issue for us, we need to be able to scan 200-300 times a row with pauses of 2-3 seconds with the pause variable. You help us a lot with your plugins thank you for everything and your future help !

[kriefsacha]

Okay so @triniwiz @NathanWalker , we really had to progress about this issue so we did some debug of the code. And we found some things that can help.

• On the file mlkit/mlkit-cameraview.android.ts, we saw that the resolution of the pictureSize is too big. it was around 3200/4600. So we tried to put the same resolution of the preview by doing line 156:

        const pictureSize = sizePair.previewSize;
        const previewSize = sizePair.previewSize;

And then line 149 instead of 1400,1200 we puted 700,600.

With this, we achieved from around 30-50 scans in a row before to around 200 !

We think by doing this garbage collector had more time to do his job.

• On the same file line 386 there is the function

createPreviewBuffer

By doing some console logs we saw that one of the two errors (i will speak about the second one after) it’s from here. Because it’s called on try and catch the app doesnt crash but it freeze. But we didn’t managed to see why this function was doing an error and where in the function.

• On the file mlkit/barcodescanning/index.android.ts line 62 there is the function

createSuccessListener

on this function line 72 you take the data of the last vision to after that just returning the width and height. The second error (the one we had from the beginning you can see on the stack trace:

System.err: at com.google.firebase.ml.vision.common.FirebaseVisionImage.getBitmap(Unknown Source:46)

) is from this. There is no try and catch there so that’s why the app was crashing. We puted in comment this part, we wasn’t returning the image anymore and then we didn’t have the error. We had just the first error of the buffer.

We actually didn’t understand why do someone need to receive the height and width of the result, and even so, maybe make it optional or something.

So that’s what we found. One way to make the error coming later, and two places where the error can come from. If it can help and we would be glad to have your opinion on all this.

[kriefsacha]

Hi @triniwiz @NathanWalker we don’t have any news for a while now…

Are you working on this please ?

Thank you