Add crop functionality to BrowserCodeReader

You often don’t want to scan whole picture. In our case datamatrix is really small, so we want to crop and scan only the center of the picture.

I propose to add additional configuration option to BrowserCodeReader allowing to pass crop options to drawImage method: what should be the size of cropped picture and what would be its position in larger picture

[bburns]

I got this working with React - it copies a crop of the video feed onto a canvas, then copies the canvas into an image element, then feeds that to the barcode reader…

Tested on MacOS Chrome so far -

// scanner video feed
// see https://github.com/zxing-js/library

import React from 'react'
import './styles.scss'
import { BrowserBarcodeReader } from '@zxing/library'

const timeout = 1000 // time between frames
const scale = 0.5 // size of crop frame

let barcodeReader
let videoStream

// user clicked on the camera button - bring up scanner window.
export async function openScanner(onFoundBarcode) {

  barcodeReader = new BrowserBarcodeReader()

  showScanner()

  // get html elements
  const video = document.querySelector('#scanner-video video')
  const canvas = document.querySelector('#scanner-canvas')
  const img = document.querySelector('#scanner-image')
  const frame = document.querySelector('#scanner-frame')

  // turn on the video stream
  const constraints = { video: true }
  navigator.mediaDevices.getUserMedia(constraints).then(stream => {
    videoStream = stream

    // handle play callback
    video.addEventListener('play', () => {
      // get video's intrinsic width and height, eg 640x480,
      // and set canvas to it to match.
      canvas.width = video.videoWidth
      canvas.height = video.videoHeight

      // set position of orange frame in video
      frame.style.width = video.clientWidth * scale + 'px'
      frame.style.height = video.clientHeight * scale + 'px'
      frame.style.left =
        (window.innerWidth - video.clientWidth * scale) / 2 + 'px'
      frame.style.top =
        (window.innerHeight - video.clientHeight * scale) / 2 + 'px'

      // start the barcode reader process
      scanFrame()
    })

    video.srcObject = stream
  })

  function scanFrame() {
    if (videoStream) {
      // copy the video stream image onto the canvas
      canvas.getContext('2d').drawImage(
        video,
        // source x, y, w, h:
        (video.videoWidth - video.videoWidth * scale) / 2,
        (video.videoHeight - video.videoHeight * scale) / 2,
        video.videoWidth * scale,
        video.videoHeight * scale,
        // dest x, y, w, h:
        0,
        0,
        canvas.width,
        canvas.height
      )
      // convert the canvas image to an image blob and stick it in an image element
      canvas.toBlob(blob => {
        const url = URL.createObjectURL(blob)
        // when the image is loaded, feed it to the barcode reader
        img.onload = async () => {
          barcodeReader
            // .decodeFromImage(img) // decodes but doesn't show img
            .decodeFromImage(null, url)
            .then(found) // calls onFoundBarcode with the barcode string
            .catch(notfound)
            .finally(releaseMemory)
          img.onload = null
          setTimeout(scanFrame, timeout) // repeat
        }
        img.src = url // load the image blob
      })
    }
  }

  function found(result) {
    onFoundBarcode(result.text)
    closeScanner()
  }

  function notfound(err) {
    if (err.name !== 'NotFoundException') {
      console.error(err)
    }
  }

  function releaseMemory() {
    URL.revokeObjectURL(img.url) // release image blob memory
    img.url = null
  }
}

export function closeScanner() {
  if (videoStream) {
    videoStream.getTracks().forEach(track => track.stop()) // stop webcam feed
    videoStream = null
  }
  hideScanner()
  barcodeReader.reset()
}

function showScanner() {
  document.querySelector('.scanner').classList.add('visible')
}

function hideScanner() {
  document.querySelector('.scanner').classList.remove('visible')
}

export default function Scanner() {
  return (
    <div className="scanner">
      <div id="scanner-video">
        <video autoPlay playsInline></video>
      </div>
      <div id="scanner-frame"></div>
      <canvas id="scanner-canvas"></canvas>
      <img id="scanner-image" src="" alt="" />
      <button id="scanner-close" onClick={closeScanner}>
        Close
      </button>
    </div>
  )
}

.scanner {
  height: 100%;
  display: none;
}
.scanner.visible {
  display: block;
}

#scanner-video {
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  right: 0;
  margin: auto;
  z-index: 1;
  background: rgba(0, 0, 0, 0.7);
  z-index: 1;
}

#scanner-video video {
  object-fit: initial;
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  right: 0;
  margin: auto;
  height: 100%;
  width: auto;
  z-index: 2;
  @media (orientation: portrait) {
    width: 100%;
    height: auto;
  }
}

#scanner-frame {
  position: absolute;
  margin: auto;
  box-shadow: 0 0 0 9999px rgba(0, 0, 0, 0.5);
  border: 2px solid orange;
  z-index: 3;
}

#scanner-canvas {
  display: none;
  width: 100%;
  margin: auto;
  z-index: 4;
}

#scanner-image {
  display: none;
  width: 100%;
  margin: auto;
  z-index: 10;
}

#scanner-close {
  position: absolute;
  top: 0;
  left: 0;
  z-index: 30;
  cursor: pointer;
}
#scanner-close:hover {
  background: #aaa;
}

[mpodlasin]

We make a crop mask with css. Here is a picture of our app:

Only stuff inside white rectangle is passed to algorithm. Since area is 4 times smaller, algorithm has much better results, than if we would pass whole picture. Scanning apps very often have masks like this.

If programmer could pass crop data to reader, he could use the same data to calculate and create mask of proper size.

The only other way to achieve this would be to write custom BrowserCodeReader, which we obviously would like to avoid.