⚡ Announcing the Capacitor ML Kit Barcode Scanning Plugin

Today I'm very excited to introduce you to the brand new Capacitor ML Kit Barcode Scanning plugin.
This plugin is part of the new Capacitor ML Kit project by Capawesome, which aims to bring the powerful ML Kit SDKs¹ to Capacitor.
The plugin allows you to scan and decode various types of barcodes, including QR codes² and UPC codes.
For a complete list of supported barcodes, see BarcodeFormat.
The scanning is done directly on the device and does not require a network connection.
The plugin supports Android and iOS, and it allows multiple barcodes to be scanned at once.
It also has torch and autofocus support, and an optional ready-to-use interface without the need for webview customizations.

Let's take a quick look at the Barcode Scanning Plugin API and how you can scan and decode barcodes.

Installation

First you need to install the package and sync your Capacitor project:

npm install @capacitor-mlkit/barcode-scanning
npx cap sync

Android

On Android, this plugin requires the following permissions be added to your AndroidManifest.xml (usually android/app/src/main/AndroidManifest.xml) before or after the application tag:

<!-- To get access to the camera. -->
<uses-permission android:name="android.permission.CAMERA" />
<!-- To get access to the flashlight. -->
<uses-permission android:name="android.permission.FLASHLIGHT"/>

You also need to add the following meta data in the application tag in your AndroidManifest.xml:

<meta-data android:name="com.google.mlkit.vision.DEPENDENCIES" android:value="barcode_ui"/>

iOS

On iOS, add the NSCameraUsageDescription key to the Info.plist file (usually ios/App/App/Info.plist), which tells the user why the app needs to use the camera:

<key>NSCameraUsageDescription</key>
<string>The app enables the scanning of various barcodes.</string>

Usage

Let's see the plugin in action.

Request permissions

In order to be able to scan barcodes, we first need the camera permissions.
We can easily request them via the plugin:

import { BarcodeScanner } from "@capacitor-mlkit/barcode-scanning";

const requestPermissions = async () => {
  await BarcodeScanner.requestPermissions();
};

In addition, you can use the method isSupported() to check whether the device has a camera:

import { BarcodeScanner } from "@capacitor-mlkit/barcode-scanning";

const isSupported = async () => {
  await BarcodeScanner.isSupported();
};

Scan barcode with ready-to-use interface

Now that you have requested the permissions, you can scan your first barcode.
To make the first scan as easy as possible and not require any WebView customization, you use thescan() method, which provides a ready-to-use interface.
By choosing a barcode format, we can improve the speed of the barcode scanner.
In this example we are only looking for QR codes² and return the rawValue of the first QR code² found:

import {
  BarcodeScanner,
  BarcodeFormat,
} from "@capacitor-mlkit/barcode-scanning";

const scan = async () => {
  const { barcodes } = await BarcodeScanner.scan({
    formats: [BarcodeFormat.QrCode],
  });
  return barcodes[0].rawValue;
};

Scan barcode with WebView customizations

If you want to design the user interface yourself or scan several barcodes in a row, you need the methods startScan(...) and stopScan().
The camera is visible behind the WebView during scanning.
However, this means that you have to hide all elements that should not be visible.
In this case we set a class barcode-scanning-active, which then contains certain CSS rules (see below) for our app.
You also need to add a barcodeScanned listener so that you are notified of detected barcodes.

import { BarcodeScanner } from "@capawesome-team/capacitor-barcode-scanner";

const startScan = async () => {
  // Hide all elements in the WebView
  document.querySelector("body")?.classList.add("barcode-scanning-active");

  // Add the `barcodeScanned` listener
  const listener = await BarcodeScanner.addListener(
    "barcodeScanned",
    async (result) => {
      // Print the found barcode to the console
      console.log(result.barcode);
    },
  );

  // Start the barcode scanner
  await BarcodeScanner.startScan();
};

const stopScan = async () => {
  // Make all elements in the WebView visible again
  document.querySelector("body")?.classList.add("barcode-scanning-active");

  // Remove all listeners
  await BarcodeScanner.removeAllListeners();

  // Stop the barcode scanner
  await BarcodeScanner.stopScan();
};

An example of the CSS class barcode-scanning-active with Ionic could be:

// Hide all elements
body.barcode-scanning-active {
  visibility: hidden;
  --background: transparent;
  --ion-background-color: transparent;
}

// Show only the barcode scanner modal
.barcode-scanning-modal {
  visibility: visible;
}

@media (prefers-color-scheme: dark) {
  .barcode-scanning-modal {
    --background: transparent;
    --ion-background-color: transparent;
  }
}

An example of the CSS class barcode-scanning-active without Ionic could be:

// Hide all elements
body.barcode-scanning-active {
  visibility: hidden;
}

// Show only the barcode scanner modal
.barcode-scanning-modal {
  visibility: visible;
}

???+ tip

If you can't see the camera view, make sure **all elements** in the DOM are not visible or have a transparent background to debug the issue.

Read barcode from image

Last but not least, you have the option of scanning barcodes from an image you have already taken.
All you need is the file path to the image.
You can get the file path, for example, if the user selects an image using the File Picker Plugin.
The file path is passed to the method readBarcodesFromImage(...), which then returns the detected barcodes:

import {
  BarcodeScanner,
  BarcodeFormat,
} from "@capacitor-mlkit/barcode-scanning";
import { FilePicker } from "@capawesome/capacitor-file-picker";

const pickImage = async () => {
  const { files } = await FilePicker.pickImages({
    multiple: true,
  });
  return files[0];
};

const scan = async () => {
  const pickedImage = await pickImage();
  const { barcodes } = await BarcodeScanner.readBarcodesFromImage({
    formats: [BarcodeFormat.QrCode],
    path: pickedImage.path,
  });
  return barcodes[0].rawValue;
};

Demo App

Feel free to download our demo app to see the plugin in action:

1. Clone the repository:

   git clone https://github.com/robingenz/capacitor-mlkit-plugin-demo.git

1. Change to the root directory:

   cd capacitor-mlkit-plugin-demo

1. Install all dependencies:

   npm i

1. Prepare and launch the Android app:

   npx ionic cap sync android
   npx ionic cap run android

1. Prepare and launch the iOS app:

   npx ionic cap sync ios
   npx ionic cap run ios

Closing Thoughts

Be sure to check out our API Reference to see what else you can do with this plugin.
If you don't want to miss any future updates, feel free to follow us on Twitter.
A big thank you to all the sponsors who make these projects possible!

1. This project is not affiliated with, endorsed by, sponsored by, or approved by Google LLC or any of their affiliates or subsidiaries. ↩

2. QR Code is a registered trademark of DENSO WAVE INCORPORATED. ↩