Xiao Ling

Posted on Jan 25 • Originally published at dynamsoft.com

Empowering .NET MAUI Android Apps with Document and MRZ Detection

#dotnet #maui #android #programming

Last week, we enhanced the .NET MAUI library Capture.Vision.Maui by adding document and MRZ (Machine-Readable Zone) detection capabilities for the Windows platform. We also enabled our .NET MAUI application to scan barcodes, documents, and MRZs in Windows. This week, we will further enhance the library by adding Android support. We will outline the entire process, from the Android AAR package to the .NET library package, and finally to the .NET MAUI library package. After upgrading the .NET MAUI library with integrated, platform-specific code for Android, your .NET MAUI application will be able to detect QR codes, documents, and MRZs from camera images without modifying any C# code.

Demo: Scanning QR Codes, Documents, and MRZs in .NET MAUI Android Applications

NuGet Package

https://www.nuget.org/packages/Capture.Vision.Maui

.NET MRZ SDK for Android

Download https://github.com/yushulx/dotnet-mrz-sdk and create a new Android Java Library Binding project in the project folder, using the same name as the desktop project.
Add DynamsoftCore.aar, DynamsoftLabelRecognizer.aar and MRZLib-release.aar to the project. Note: MRZLib-release.aar is generated by the MRZLib project.
Copy MrzParser.cs, MrzResult.cs, and MrzScanner.cs from the desktop project to the Android Java Library Binding project.

Refactor MrzScanner.cs to utilize the Dynamsoft Label Recognizer Android library.

using Com.Dynamsoft.Core;
using Com.Dynamsoft.Dlr;

namespace Dynamsoft
{
    public class MrzScanner
    {
        private MRZRecognizer? recognizer;
        ...

        public class LicenseVerificationListener : Java.Lang.Object, ILicenseVerificationListener
        {
            public void LicenseVerificationCallback(bool isSuccess, CoreException ex)
            {
                if (!isSuccess)
                {
                    throw new Exception(ex.ToString());
                }
            }
        }

        public static void InitLicense(string license, object? context = null)
        {
            if (context == null) { return; }

            LicenseManager.InitLicense(license, (Android.Content.Context)context, new LicenseVerificationListener());
        }

        private MrzScanner()
        {
            recognizer = new MRZRecognizer();
        }

        public static MrzScanner Create()
        {
            return new MrzScanner();
        }

        ~MrzScanner()
        {
            Destroy();
        }

        public void Destroy()
        {
            recognizer = null;
        }

        public static string? GetVersionInfo()
        {
            return MRZRecognizer.Version;
        }

        public Result[]? DetectFile(string filename)
        {
            if (recognizer == null) return null;

            DLRResult[]? mrzResult = recognizer.RecognizeFile(filename);
            return GetResults(mrzResult);
        }

        public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
        {
            if (recognizer == null) return null;

            ImageData imageData = new ImageData()
            {
                Bytes = buffer,
                Width = width,
                Height = height,
                Stride = stride,
                Format = (int)format,
            };
            DLRResult[]? mrzResult = recognizer.RecognizeBuffer(imageData);
            return GetResults(mrzResult);
        }

        private Result[]? GetResults(DLRResult[]? mrzResult)
        {
            if (mrzResult != null && mrzResult[0].LineResults != null)
            {
                DLRLineResult[] lines = mrzResult[0].LineResults.ToArray();
                Result[] result = new Result[lines.Length];
                for (int i = 0; i < lines.Length; i++)
                {
                    result[i] = new Result()
                    {
                        Confidence = lines[i].Confidence,
                        Text = lines[i].Text ?? "",
                        Points = new int[8]
                        {
                            lines[i].Location.Points[0].X,
                            lines[i].Location.Points[0].Y,
                            lines[i].Location.Points[1].X,
                            lines[i].Location.Points[1].Y,
                            lines[i].Location.Points[2].X,
                            lines[i].Location.Points[2].Y,
                            lines[i].Location.Points[3].X,
                            lines[i].Location.Points[3].Y
                        }
                    };
                }

                return result;
            }

            return null;
        }
    }
}

Generate a MrzScannerSDK.nuspec file at the root directory.

<?xml version="1.0" encoding="utf-8"?>
<package>
  <metadata>
    <id>MrzScannerSDK</id>
    <version>1.3.5</version>
    <title>MRZ Scanner SDK</title>
    <authors>yushulx</authors>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <license type="expression">MIT</license>
    <readme>README.md</readme>
    <!-- <icon>icon.png</icon> -->
    <projectUrl>https://github.com/yushulx/dotnet-mrz-sdk</projectUrl>
    <description>The MRZ Scanner SDK is a .NET wrapper for Dynamsoft Label Recognizer,
      supporting x64 Windows, x64 Linux and Android.</description>
    <releaseNotes>Fixed build condition for Windows.</releaseNotes>
    <copyright>$copyright$</copyright>
    <tags>MRZ;mrz-scan;machine-readable-zone;mrz-detection;passport;visa;id-card;travel-document</tags>
  </metadata>
  <files>
    <file src="README.md" target="" />
    <file src="LICENSE.txt" target="" />

    <!-- Desktop -->
    <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
    <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
    <file src="desktop\bin\Release\net7.0\MrzScannerSDK.dll" target="lib\net7.0" />
    <file src="desktop\MrzScannerSDK.targets" target="build" />
    <file src="desktop\model\**\*.*" target="model" />

    <!-- Android -->
    <file src="android\sdk\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />

  </files>
</package>

Build the desktop and Android projects, then bundle them into a single NuGet package.

cd desktop
dotnet build --configuration Release

cd android
dotnet build --configuration Release

nuget pack .\MrzScannerSDK.nuspec

.NET Document SDK for Android

Download https://github.com/yushulx/dotnet-document-scanner-sdk and create a new Android Java Library Binding project in the project folder, using the same name as the desktop project.
Add DynamsoftCore.aar, DynamsoftDocumentNormalizer.aar, DynamsoftImageProcessing.aar and DynamsoftIntermediateResult.aar to the project.
Copy DocumentScanner.cs from the desktop project to the Android Java Library Binding project.

Refactor the DocumentScanner.cs to utilize the Dynamsoft Document Normalizer Android library.

using Com.Dynamsoft.Core;
using Com.Dynamsoft.Ddn;

namespace Dynamsoft
{
    public class DocumentScanner
    {
        private DocumentNormalizer normalizer;

        public class NormalizedImage
        {
            public int Width;
            public int Height;
            public int Stride;
            public ImagePixelFormat Format;
            public byte[] Data = new byte[0];
        }

        public static string GetVersionInfo()
        {
            return DocumentNormalizer.Version;
        }

        ...

        public class LicenseVerificationListener : Java.Lang.Object, ILicenseVerificationListener
        {
            public void LicenseVerificationCallback(bool isSuccess, CoreException ex)
            {
                if (!isSuccess)
                {
                    throw new Exception(ex.ToString());
                }
            }
        }

        public static void InitLicense(string license, object? context = null)
        {
            if (context == null) { return; }

            LicenseManager.InitLicense(license, (Android.Content.Context)context, new LicenseVerificationListener());
        }

        private DocumentScanner()
        {
            normalizer = new DocumentNormalizer();
        }

        public static DocumentScanner Create()
        {
            return new DocumentScanner();
        }

        public void SetParameters(string parameters)
        {
            try
            {
                normalizer.InitRuntimeSettingsFromString(parameters);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
        }

        public Result[]? DetectFile(string filename)
        {
            DetectedQuadResult[]? results = normalizer.DetectQuad(filename);
            return GetResults(results);
        }

        public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
        {
            ImageData imageData = new ImageData()
            {
                Bytes = buffer,
                Width = width,
                Height = height,
                Stride = stride,
                Format = (int)format,
            };
            DetectedQuadResult[]? results = normalizer.DetectQuad(imageData);
            return GetResults(results);
        }

        private Result[]? GetResults(DetectedQuadResult[]? results)
        {
            if (results == null) return null;

            var result = new Result[results.Length];
            for (int i = 0; i < results.Length; i++)
            {
                DetectedQuadResult tmp = results[i];
                Quadrilateral quad = tmp.Location;
                result[i] = new Result()
                {
                    Confidence = tmp.ConfidenceAsDocumentBoundary,
                    Points = new int[8]
                    {
                        quad.Points[0].X, quad.Points[0].Y,
                        quad.Points[1].X, quad.Points[1].Y,
                        quad.Points[2].X, quad.Points[2].Y,
                        quad.Points[3].X, quad.Points[3].Y,
                    }
                };
            }

            return result;
        }

        public NormalizedImage NormalizeFile(string filename, int[] points)
        {
            Quadrilateral quad = new Quadrilateral();
            quad.Points = new Android.Graphics.Point[4];
            quad.Points[0] = new Android.Graphics.Point(points[0], points[1]);
            quad.Points[1] = new Android.Graphics.Point(points[2], points[3]);
            quad.Points[2] = new Android.Graphics.Point(points[4], points[5]);
            quad.Points[3] = new Android.Graphics.Point(points[6], points[7]);
            NormalizedImageResult? result = normalizer.Normalize(filename, quad);
            return GetNormalizedImage(result);
        }

        public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points)
        {
            ImageData imageData = new ImageData()
            {
                Bytes = buffer,
                Width = width,
                Height = height,
                Stride = stride,
                Format = (int)format,
            };

            Quadrilateral quad = new Quadrilateral();
            quad.Points = new Android.Graphics.Point[4];
            quad.Points[0] = new Android.Graphics.Point(points[0], points[1]);
            quad.Points[1] = new Android.Graphics.Point(points[2], points[3]);
            quad.Points[2] = new Android.Graphics.Point(points[4], points[5]);
            quad.Points[3] = new Android.Graphics.Point(points[6], points[7]);
            NormalizedImageResult? result = normalizer.Normalize(imageData, quad);
            return GetNormalizedImage(result);
        }

        private NormalizedImage GetNormalizedImage(NormalizedImageResult? result)
        {
            NormalizedImage normalizedImage = new NormalizedImage();
            if (result != null)
            {
                ImageData imageData = result.Image;
                normalizedImage.Width = imageData.Width;
                normalizedImage.Height = imageData.Height;
                normalizedImage.Stride = imageData.Stride;
                normalizedImage.Format = (ImagePixelFormat)imageData.Format;
                normalizedImage.Data = imageData.Bytes.ToArray();
            }
            return normalizedImage;
        }
    }
}

Generate a MrzScannerSDK.nuspec file in the root directory.

<?xml version="1.0" encoding="utf-8"?>
<package>
    <metadata>
        <id>DocumentScannerSDK</id>
        <version>1.2.0</version>
        <title>Document Scanner SDK</title>
        <authors>yushulx</authors>
        <requireLicenseAcceptance>false</requireLicenseAcceptance>
        <license type="expression">MIT</license>
        <readme>README.md</readme>
        <!-- <icon>icon.png</icon> -->
        <projectUrl>https://github.com/yushulx/dotnet-document-scanner-sdk</projectUrl>
        <description>The Document Scanner SDK is a .NET wrapper for Dynamsoft Document Normalizer,
            supporting x64 Windows, x64 Linux and Android.</description>
        <releaseNotes>Added support for Android.</releaseNotes>
        <copyright>$copyright$</copyright>
        <tags>document;document-scan;edge-detection;document-detection</tags>
    </metadata>
    <files>
        <file src="README.md" target="" />
        <file src="LICENSE.txt" target="" />

        <!-- Desktop -->
        <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
        <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
        <file src="desktop\bin\Release\net7.0\DocumentScannerSDK.dll" target="lib\net7.0" />

        <!-- Android -->
        <file src="android\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />

    </files>
</package>

Build the desktop and Android projects, then bundle them into a single NuGet package.

cd desktop
dotnet build --configuration Release

cd android
dotnet build --configuration Release

nuget pack .\DocumentScannerSDK.nuspec

Troubleshooting Error XA4215 When Building a .NET MAUI Library Project

When building a .NET MAUI library project that includes dependencies on the .NET MRZ SDK and the .NET Document SDK, you may encounter the following error:

Severity    Code    Description Project File    Line    Suppression State   Details
Error   XA4215    `mono.com.dynamsoft.core.LicenseVerificationListenerImplementor` generated by: Com.Dynamsoft.Core.ILicenseVerificationListenerImplementor, MrzScannerSDK, Version=0.1.4.0, Culture=neutral, PublicKeyToken=null   Capture.Vision.Maui.Example C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.95\tools\Xamarin.Android.Common.targets    1476        
Error   XA4215    `mono.com.dynamsoft.core.LicenseVerificationListenerImplementor` generated by: Com.Dynamsoft.Core.ILicenseVerificationListenerImplementor, DocumentScannerSDK, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null  Capture.Vision.Maui.Example C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.95\tools\Xamarin.Android.Common.targets    1476

The error XA4215 you're encountering in your .NET MAUI project indicates a conflict between two assemblies trying to generate the same class, mono.com.dynamsoft.core.LicenseVerificationListenerImplementor. To resolve the issue, a feasible workaround is to merge the two packages into one.

Merging .NET MRZ SDK and .NET Document SDK into One .NET Library

Create a Class Library project and an Android Java Library Binding project in the same folder, ensuring both projects share the same name, e.g., CaptureVision. The Class Library project will house .NET code and libraries for desktop platforms (Windows and Linux), whereas the Android Java Library Binding project will house .NET code and libraries for Android.
Copy all Android .aar packages and C# files from MrzScannerSDK and DocumentScannerSDK android folder to the Android Java Library Binding project.

Copy MRZ model files, *.so files, *.dll files, and C# files from MrzScannerSDK and DocumentScannerSDK desktop folder to the Class Library project, maintaining the original folder structure.

├── model
├── lib
    ├── linux
    ├── win
├── CaptureVision.csproj
├── CaptureVision.targets
├── DocumentScaner.cs
├── MrzParser.cs
├── MrzResult.cs
├── MrzScanner.cs

Generate a CaptureVision.nuspec file in the root directory.

<?xml version="1.0" encoding="utf-8"?>
<package>
    <metadata>
        <id>CaptureVision</id>
        <version>1.0.1</version>
        <title>Capture Vision SDK</title>
        <authors>yushulx</authors>
        <requireLicenseAcceptance>false</requireLicenseAcceptance>
        <license type="expression">MIT</license>
        <readme>README.md</readme>
        <!-- <icon>icon.png</icon> -->
        <projectUrl>https://github.com/yushulx/Capture-Vision</projectUrl>
        <description>This is a package that is a compound of DocumentScannerSDK and MrzScannerSDK.</description>
        <releaseNotes>Merged DocumentScannerSDK and MrzScannerSDK into one package.</releaseNotes>
        <copyright>$copyright$</copyright>
        <tags>
            MRZ;Android;passport;id-card;visa;machine-readable-zone;document;document-scan;edge-detection;document-detection</tags>
    </metadata>
    <files>
        <file src="README.md" target="" />
        <file src="LICENSE.txt" target="" />

        <!-- Desktop -->
        <file src="desktop\lib\win\**\*.*" target="runtimes\win-x64\native" />
        <file src="desktop\lib\linux\**\*.*" target="runtimes\linux-x64\native" />
        <file src="desktop\bin\Release\net7.0\CaptureVision.dll" target="lib\net7.0" />
        <file src="desktop\CaptureVision.targets" target="build" />
        <file src="desktop\model\**\*.*" target="model" />

        <!-- Android -->
        <file src="android\bin\Release\net7.0-android\**\*.*" target="lib\net7.0-android33.0" />

    </files>
</package>

Build the library projects respectively and pack them into a single NuGet package.

cd desktop
dotnet build --configuration Release

cd android
dotnet build --configuration Release

nuget pack .\CaptureVision.nuspec

Adding Android Platform-Specific Code to .NET MAUI Library

After generating the NuGet package, incorporate it as a dependency into your .NET MAUI library project.

<ItemGroup Condition="'$(TargetFramework)' == 'net7.0-android'">
    <PackageReference Include="CaptureVision " Version="1.0.1" />
</ItemGroup>

<ItemGroup Condition="$([MSBuild]::IsOSPlatform('windows'))">
    <PackageReference Include="CaptureVision " Version="1.0.1" />
</ItemGroup>

Next, navigate to Platforms/Android/NativeCameraView.cs within your project, and insert the following code snippet:

private BarcodeQRCodeReader barcodeReader;
private MrzScanner mrzScanner;
private DocumentScanner documentScanner;

public NativeCameraView(Context context, CameraView cameraView) : base(context)
{
    ...

    barcodeReader = BarcodeQRCodeReader.Create();
    mrzScanner = MrzScanner.Create();
    documentScanner = DocumentScanner.Create();
}

private void StartPreview()
{
    ...
    backgroundHandler = new Handler(backgroundThread.Looper);
    frameListener = new ImageAvailableListener(cameraView, barcodeReader, mrzScanner, documentScanner);
    ...
}

class ImageAvailableListener : Java.Lang.Object, ImageReader.IOnImageAvailableListener
{
    private readonly CameraView cameraView;
    private BarcodeQRCodeReader barcodeReader;
    private MrzScanner mrzScanner;
    private DocumentScanner documentScanner;

    public ImageAvailableListener(CameraView camView, BarcodeQRCodeReader barcodeReader, MrzScanner mrzScanner, DocumentScanner documentScanner)
    {
        cameraView = camView;
        this.barcodeReader = barcodeReader;
        this.mrzScanner = mrzScanner;
        this.documentScanner = documentScanner;
    }

    public void OnImageAvailable(ImageReader reader)
    {
        try
        {
            var image = reader?.AcquireLatestImage();
            ...

            if (cameraView.EnableDocumentDetect)
            {
                DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytes, width, height, nPixelStride * nRowStride, DocumentScanner.ImagePixelFormat.IPF_GRAYSCALED);
                DocumentResult documentResults = new DocumentResult();
                ...
                cameraView.NotifyResultReady(documentResults, width, height);
            }

            if (cameraView.EnableMrz)
            {
                MrzResult mrzResults = new MrzResult();
                try
                {
                    MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytes, width, height, nPixelStride * nRowStride, MrzScanner.ImagePixelFormat.IPF_GRAYSCALED);
                    ...
                }

                catch (Exception ex)
                {
                    System.Diagnostics.Debug.WriteLine(ex.Message);
                }
                cameraView.NotifyResultReady(mrzResults, width, height);
            }
        }
        catch (Exception ex)
        {
            System.Diagnostics.Debug.WriteLine(ex.Message);
        }
    }
}

Upgrading Capture.Vision.Maui to Scan QR Code, Document, and MRZ in .NET MAUI Android Applications

In the .NET MAUI application project, upgrade the Capture.Vision.Maui library to its latest version. After upgrading, rebuild the project and execute it on an Android device.

QR Code Scan

Document Edge Detection

MRZ Recognition

Source Code

https://github.com/yushulx/Capture-Vision-Maui

DEV Community

Empowering .NET MAUI Android Apps with Document and MRZ Detection

Demo: Scanning QR Codes, Documents, and MRZs in .NET MAUI Android Applications

NuGet Package

.NET MRZ SDK for Android

.NET Document SDK for Android

Troubleshooting Error XA4215 When Building a .NET MAUI Library Project

Merging .NET MRZ SDK and .NET Document SDK into One .NET Library

Adding Android Platform-Specific Code to .NET MAUI Library

Upgrading Capture.Vision.Maui to Scan QR Code, Document, and MRZ in .NET MAUI Android Applications

Source Code

Top comments (0)

Read next

Decoding JavaScript Variables: A Comprehensive Overview

I fine-tuned my model on a new programming language. You can do it too! 🚀

Programação não é fácil, mas ao mesmo tempo é fascinante

How to have AI onboard your users (Next.js, OpenAI, CopilotKit)