Highlight scan barcodes for better UX

While scanning barcodes, it is possible to retrieve a barcode object with either boundingBox for Android or cornerPoints for iOS.

With these properties, it seems possible to get the right positioning of the barcode in order to highlight it for better user experience. For instance, while scanning multiple barcodes, it can be useful to add an emoji or show the borders of the barcodes already scanned.

The issue is that I am not able to display correctly the borders of scanned barcodes. I am trying to adapt boundingBox and cornerPoints to use these values in a more classic way to display absolute elements with left and top properties but it does not fit perfectly with the real barcode borders. Is there a way to use boundingBox or cornerPoints correctly ?

See below an example to reproduce this issue and an example of cornerPoints value.

import React, {useEffect, useState} from 'react';
import {
  View,
  Text,
  StyleSheet,
  PixelRatio,
  Platform,
} from 'react-native';
import {Camera, useCameraDevices} from 'react-native-vision-camera';

import {Colors} from 'react-native/Libraries/NewAppScreen';
import {BarcodeFormat, useScanBarcodes} from 'vision-camera-code-scanner';

const styles = StyleSheet.create({
  outer: {
    ...StyleSheet.absoluteFillObject,
    backgroundColor: Colors.darker,
    flex: 1,
  },
});

const SquareFromCoord = ({barcode}) => {
  let bottom;
  let left;
  let right;
  let top;

  const pixelRatio = PixelRatio.get();
  if (Platform.OS === 'ios') {
    const xArray = barcode.cornerPoints.map(corner => parseFloat(corner.x));
    const yArray = barcode.cornerPoints.map(corner => parseFloat(corner.y));

    left = Math.min(...xArray) / pixelRatio;
    right = Math.max(...xArray) / pixelRatio;
    bottom = Math.max(...yArray) / pixelRatio;
    top = Math.min(...yArray) / pixelRatio;
  } else {
    // Other similar logic for Android
  }

  return (
    <View
      style={{
        backgroundColor: 'rgba(0, 255, 131, 0.8)',
        borderRadius: 10,
        height: bottom - top,
        opacity: 0.8,
        width: right - left,
        position: 'absolute',
        left,
        top,
      }}
    />
  );
};

export const RnCamera = () => {
  const devices = useCameraDevices();
  const device = devices.back;

  const [frameProcessor, barcodes] = useScanBarcodes([BarcodeFormat.QR_CODE]);
  
  const [permissions, setPermissions] = useState(false);

  const handlePermission = async () => {
    const cameraPermission = await Camera.getCameraPermissionStatus();
    if (cameraPermission !== 'authorized') {
      const askPermissionRes = await Camera.requestCameraPermission();
      setPermissions(askPermissionRes === 'authorized');
    } else {
      setPermissions(true);
    }
  };

  useEffect(() => {
    handlePermission();
  }, []);

  if (!device || !permissions) {
    return <Text>Loading</Text>;
  }

  return (
    <View style={styles.outer}>
      <Camera
        style={StyleSheet.absoluteFill}
        device={device}
        isActive={true}
        enableZoomGesture
        frameProcessor={frameProcessor}
        frameProcessorFps={5}
      />
      {barcodes.map((barcode, idx) => (
        <SquareFromCoord key={idx} barcode={barcode} />
      ))}
    </View>
  );
};

Example of cornerPoints while scanning a QR code with an iPad Air 2 (window dimensions = 677:335) : "cornerPoints": [{"x": 209.60528564453125, "y": 647.1997680664062}, {"x": 331.0139465332031, "y": 655.4308471679688}, {"x": 330.98651123046875, "y": 780.6898193359375}, {"x": 209.5778350830078, "y": 772.458740234375}]

[kirill3333]

@qboisson I’ve changed it a little and it looks very close to reality

const WINDOW_HEIGHT = Dimensions.get(‘window’).height; const WINDOW_WIDTH = Dimensions.get(‘window’).width;

...
const frameProcessor = useFrameProcessor(frame => {
    'worklet';
    const detectedBarcodes = scanBarcodes(frame, [BarcodeFormat.ALL_FORMATS]);
    const barcode = detectedBarcodes[0];
    if (barcode) {
      const cornerPoints = barcode.cornerPoints;
      const xRatio = frame.width / WINDOW_WIDTH;
      const yRatio = frame.height / WINDOW_HEIGHT;
      if (Platform.OS === 'ios') {
        const xArray = cornerPoints.map(corner => parseFloat(corner.x));
        const yArray = cornerPoints.map(corner => parseFloat(corner.y));
        const resultPoints = {
          left: Math.min(...xArray) / xRatio,
          right: Math.max(...xArray) / xRatio,
          bottom: Math.max(...yArray) / yRatio,
          top: Math.min(...yArray) / yRatio,
        };
        runOnJS(setPointsOfInterest)(resultPoints);
      } else {
        // Other similar logic for Android
      }
      runOnJS(setBarcodes)(detectedBarcodes);
    } else {
      runOnJS(setPointsOfInterest)(null);
    }
  }, []);
...

[mgcrea]

Unfortunately it’s more complex than the current given solutions as it actually depends on the camera view size (that can be equal to the window when in fullscreen), and there is also some cropping after the resize that you have to take into account to setup proper offsets.

Currently working on a solution that covers all cases, @rodgomesc would you be interested in a PR adding this highlight feature?