DEV Community

Cover image for ML Kit in React Native (Part 1)
Dima Portenko
Dima Portenko

Posted on • Updated on

ML Kit in React Native (Part 1)

Machine learning isn't a science problem anymore. It's a common part of mobile apps nowadays. I found Google ML Kit is a quite interesting solution and decided to cover how we can use it in React Native. Since there is no dedicated library we will use native modules to wire up the JavaScript side with iOS and Android SDKs.

This tutorial will cover two topics - native modules basics for iOS and Android and Text Recognition API from the Google ML Kit. I also have it in video format on my YouTube channel

ML Kit in React Native YouTube playlist


What are we building?

Final look gif video


Boilerplate project

We are going to recognise text from the images. So I prepared project boilerplate which allow get image from Camera or Gallery. You can download it from here. We're using react-native-image-picker to select image and get its uri.


Android Native Module

Let's start with Android. And first of all, we need to create a Native Module. To do so we need simply follow the steps in the official docs.

  1. Open the ./android/ project in the Android Studio.

    Open the ./android/ project in Android Studio

  2. Create a custom native module file ./android/app/src/main/java/com/rnmlkittutorial/mlkit/TextRecognitionModule.java

    Then add the following content:

    package com.rnmlkittutorial.mlkit;
    
    import android.util.Log;
    import com.facebook.react.bridge.ReactApplicationContext;
    import com.facebook.react.bridge.ReactContextBaseJavaModule;
    import com.facebook.react.bridge.ReactMethod;
    
    public class TextRecognitionModule extends ReactContextBaseJavaModule {
        TextRecognitionModule(ReactApplicationContext context) {
            super(context);
        }
    
        @Override
        public String getName() {
            return "TextRecognitionModule";
        }
    
        @ReactMethod
        public void recognizeImage(String url) {
            Log.d("TextRecognitionModule", "Url: " + url);
        }
    }
    

    Our class TextRecognitionModule extends ReactContextBaseJavaModule which has inner implementation required by React Native.

    Method getName returns our module which we will use in React JavaScript.

    public String getName() {
        return "TextRecognitionModule";
    }
    

    @ReactMethod annotation means this method will be accessible on the JavaScript side. And we will use recognizeImage method to pass image url to the native Java side.

    @ReactMethod
    public void recognizeImage(String url) {
        Log.d("TextRecognitionModule", "Url: " + url);
    }
    
  3. To add your Native Module to ReactPackage, first create a new Java Class named TextRecognitionPackage.java that implements ReactPackage inside the ./android/app/src/main/java/com/rnmlkittutorial/mlkit/ folder. Then add the following content:

    package com.rnmlkittutorial.mlkit;
    
    import com.facebook.react.ReactPackage;
    import com.facebook.react.bridge.NativeModule;
    import com.facebook.react.bridge.ReactApplicationContext;
    import com.facebook.react.uimanager.ViewManager;
    
    import java.util.ArrayList;
    import java.util.Collections;
    import java.util.List;
    
    public class TextRecognitionPackage implements ReactPackage {
        @Override
        public List<ViewManager> createViewManagers(ReactApplicationContext reactContext) {
            return Collections.emptyList();
        }
    
        @Override
        public List<NativeModule> createNativeModules(
            ReactApplicationContext reactContext) {
            List<NativeModule> modules = new ArrayList<>();
    
            modules.add(new TextRecognitionModule(reactContext));
    
            return modules;
        }
    }
    
    

    Inside createNativeModules method we create an instance of our module and add it to the NativeModules array which this package class contains.
    createViewManagers is responsible for UI components that we don't use. So simply return the empty list here.

  4. To register the TextRecognitionModule package, we must add TextRecognitionPackage to the list of packages returned in ReactNativeHost's getPackages() method. Open up your MainApplication.java file, which can be found in the following path: ./android/app/src/main/java/com/rnmlkittutorial/MainApplication.java

    @Override
    protected List<ReactPackage> getPackages() {
      @SuppressWarnings("UnnecessaryLocalVariable")
      List<ReactPackage> packages = new PackageList(this).getPackages();
      // Packages that cannot be autolinked yet can be added manually here, for example:
       packages.add(new TextRecognitionPackage());
      return packages;
    }
    
  5. Here we can test what we have built. Let's back to our TypeScript side and create file src/mlkit/index.ts. And put the following code into it

    import {NativeModules} from 'react-native';
    
    const {TextRecognitionModule} = NativeModules;
    
    export const recognizeImage = (url: string) => {
      return TextRecognitionModule.recognizeImage(url);
    }
    
    

    We take NativeModules from React Native which now contain our TextRecognitionModule. And we can invoke recognizeImage function we defined on the Java side. Let's go to the src/screens/ProcessImageScreen.tsx and use our module function like this

    ...
    import {recognizeImage} from '../mlkit';
    ...
    const proccessImage = async (url: string) => {
      if (url) {
        recognizeImage(url);
      }
    };
    ...
    
    
  6. Now run the android app npx react-native run-android or whatever you like. Select the image and tap Process Image.

    Select and Process Image

    Then you can back to the Android Studio and press Logcat tab. To find our log put TextRecognitionModule in the search field. This is first argument of Android Log.d function Log.d("TextRecognitionModule", "Url: " + url);

    Android Studio Locat tab screenshot

We prepared our React Native app to use Google ML Kit API on Android. Let's continue in Part 2. Source code is available here.

Top comments (2)

Collapse
 
jacquesdev profile image
Jacques de Villiers

Hello! Was part 2 ever released?

Collapse
 
dimaportenko profile image
Dima Portenko

Full version is on youtube. Just haven't found motivation to fully convert it to the text blog posts.

https://www.youtube.com/watch?v=-cB1nXlN2PU&list=PL97fL9DAn9QzKd4-exwa-vd1rNDov8wXZ