Machine learning isn't a science problem anymore. It's a common part of mobile apps nowadays. I found Google ML Kit is a quite interesting solution and decided to cover how we can use it in React Native. Since there is no dedicated library we will use native modules to wire up the JavaScript side with iOS and Android SDKs.
This tutorial will cover two topics - native modules basics for iOS and Android and Text Recognition API from the Google ML Kit. I also have it in video format on my YouTube channel
What are we building?
Boilerplate project
We are going to recognise text from the images. So I prepared project boilerplate which allow get image from Camera or Gallery. You can download it from here. We're using react-native-image-picker
to select image and get its uri.
Android Native Module
Let's start with Android. And first of all, we need to create a Native Module. To do so we need simply follow the steps in the official docs.
-
Open the
./android/
project in the Android Studio. -
Create a custom native module file
./android/app/src/main/java/com/rnmlkittutorial/mlkit/TextRecognitionModule.java
Then add the following content:
package com.rnmlkittutorial.mlkit; import android.util.Log; import com.facebook.react.bridge.ReactApplicationContext; import com.facebook.react.bridge.ReactContextBaseJavaModule; import com.facebook.react.bridge.ReactMethod; public class TextRecognitionModule extends ReactContextBaseJavaModule { TextRecognitionModule(ReactApplicationContext context) { super(context); } @Override public String getName() { return "TextRecognitionModule"; } @ReactMethod public void recognizeImage(String url) { Log.d("TextRecognitionModule", "Url: " + url); } }
Our class
TextRecognitionModule
extendsReactContextBaseJavaModule
which has inner implementation required by React Native.Method
getName
returns our module which we will use in React JavaScript.
public String getName() { return "TextRecognitionModule"; }
@ReactMethod
annotation means this method will be accessible on the JavaScript side. And we will userecognizeImage
method to pass image url to the native Java side.
@ReactMethod public void recognizeImage(String url) { Log.d("TextRecognitionModule", "Url: " + url); }
-
To add your Native Module to
ReactPackage
, first create a new Java Class namedTextRecognitionPackage.java
that implementsReactPackage
inside the./android/app/src/main/java/com/rnmlkittutorial/mlkit/
folder. Then add the following content:
package com.rnmlkittutorial.mlkit; import com.facebook.react.ReactPackage; import com.facebook.react.bridge.NativeModule; import com.facebook.react.bridge.ReactApplicationContext; import com.facebook.react.uimanager.ViewManager; import java.util.ArrayList; import java.util.Collections; import java.util.List; public class TextRecognitionPackage implements ReactPackage { @Override public List<ViewManager> createViewManagers(ReactApplicationContext reactContext) { return Collections.emptyList(); } @Override public List<NativeModule> createNativeModules( ReactApplicationContext reactContext) { List<NativeModule> modules = new ArrayList<>(); modules.add(new TextRecognitionModule(reactContext)); return modules; } }
Inside
createNativeModules
method we create an instance of our module and add it to theNativeModules
array which this package class contains.
createViewManagers
is responsible for UI components that we don't use. So simply return the empty list here. -
To register the
TextRecognitionModule
package, we must addTextRecognitionPackage
to the list of packages returned in ReactNativeHost'sgetPackages()
method. Open up yourMainApplication.java
file, which can be found in the following path:./android/app/src/main/java/com/rnmlkittutorial/MainApplication.java
@Override protected List<ReactPackage> getPackages() { @SuppressWarnings("UnnecessaryLocalVariable") List<ReactPackage> packages = new PackageList(this).getPackages(); // Packages that cannot be autolinked yet can be added manually here, for example: packages.add(new TextRecognitionPackage()); return packages; }
-
Here we can test what we have built. Let's back to our TypeScript side and create file
src/mlkit/index.ts
. And put the following code into it
import {NativeModules} from 'react-native'; const {TextRecognitionModule} = NativeModules; export const recognizeImage = (url: string) => { return TextRecognitionModule.recognizeImage(url); }
We take
NativeModules
from React Native which now contain ourTextRecognitionModule
. And we can invokerecognizeImage
function we defined on the Java side. Let's go to thesrc/screens/ProcessImageScreen.tsx
and use our module function like this
... import {recognizeImage} from '../mlkit'; ... const proccessImage = async (url: string) => { if (url) { recognizeImage(url); } }; ...
-
Now run the android app
npx react-native run-android
or whatever you like. Select the image and tapProcess Image
.Then you can back to the Android Studio and press Logcat tab. To find our log put
TextRecognitionModule
in the search field. This is first argument of AndroidLog.d
functionLog.d("TextRecognitionModule", "Url: " + url);
We prepared our React Native app to use Google ML Kit API on Android. Let's continue in Part 2
. Source code is available here.
Top comments (2)
Hello! Was part 2 ever released?
Full version is on youtube. Just haven't found motivation to fully convert it to the text blog posts.
https://www.youtube.com/watch?v=-cB1nXlN2PU&list=PL97fL9DAn9QzKd4-exwa-vd1rNDov8wXZ