Implement Google Vision in React Native apps

In this tutorial you will get know how can we integrate Google Vision API in our React Native application and get responses on text and image recognition. The app supports various features including but not limited to Object Detection, Labeling Images, Text Recognition, Face Detection using Google Vision API.

Complete source code of this tutorial can be found at ReactNativeGoogleVision

In this tutorial we will focus on some basic and major parts where many expert developers or juvenile beginners gets confused. Hence we will check each and every things in step-by-step as: —

1. What and why React Native
2. What and why Google Vision API
3. How to create react native app
4. How to get google vision API key
5. Implementation and setup
6. Working example

Step 1: — What and why React Native ?

React Native is a JavaScript framework for writing real, natively rendering mobile applications for iOS and Android. It’s based on React, Facebook’s JavaScript library for building user interfaces, but instead of targeting the browser, it targets mobile platforms. In other words: web developers can now write mobile applications that look and feel truly “native,” all from the comfort of a JavaScript library that we already know and love. Plus, because most of the code you write can be shared between platforms, React Native makes it easy to simultaneously develop for both Android and iOS.

Similar to React for the Web, React Native applications are written using a mixture of JavaScript and XML-Esque markup, known as JSX. Then, under the hood, the React Native “bridge” invokes the native rendering APIs in Objective-C (for iOS) or Java (for Android). Thus, your application will render using real mobile UI components, not webviews, and will look and feel like any other mobile application. React Native also exposes JavaScript interfaces for platform APIs, so your React Native apps can access platform features like the phone camera, or the user’s location.

React Native currently supports both iOS and Android and has the potential to expand to future platforms as well. In this tutorial, we’ll cover firebase phone authentication and CRUD operations. The vast majority of the code we write will be cross-platform. And yes: you can really use React Native to build production-ready mobile applications! Some anecdota: Facebook, Palantir, and TaskRabbit are already using it in production for user-facing applications.
 The fact that React Native actually renders using its host platform’s standard rendering APIs enables it to stand out from most existing methods of cross-platform application development, like Cordova or Ionic.

Existing methods of writing mobile applications using combinations of JavaScript, HTML, and CSS typically render using webviews. While this approach can work, it also comes with drawbacks, especially around performance. Additionally, they do not usually have access to the host platform’s set of native UI elements. When these frameworks do try to mimic native UI elements, the results usually “feel” just a little off; reverse-engineering all the fine details of things like animations takes an enormous amount of effort, and they can quickly become out of date.

Step 2: — What and why Google Vision API ?

Google Vision API is also known as Cloud Vision API.

Cloud Vision API allows developers to easily integrate vision detection features including image labeling, face, and landmark detection, optical character recognition (OCR), and tagging of explicit content, within applications.
Cloud Vision API offers us the following features to be applied on the images as follow: —

features: [

{ type: "LABEL_DETECTION" },

{ type: "LANDMARK_DETECTION" },

{ type: "FACE_DETECTION" },

{ type: "LOGO_DETECTION" },

{ type: "TEXT_DETECTION" },

{ type: "DOCUMENT_TEXT_DETECTION" },

{ type: "SAFE_SEARCH_DETECTION" },

{ type: "IMAGE_PROPERTIES" },

{ type: "CROP_HINTS" },

{ type: "WEB_DETECTION" }

],

For more information about Google Vision API, go to its official documentation here.

Brief History of Machine Learning

Arthur Samuel, a pioneer in the field of artificial intelligence and computer gaming, coined the term “Machine Learning”. He defined machine learning as — “Field of study that gives computers the capability to learn without being explicitly programmed”.
In a very layman manner, Machine Learning(ML) can be explained as automating and improving the learning process of computers based on their experiences without being actually programmed i.e. without any human assistance. The process starts with feeding good quality data and then training our machines(computers) by building machine learning models using the data and different algorithms.

ML is one of the most impressive technologies that one would have ever come across.

Step 3: — How to create react native app ?

Now till here, i expect all of you have created your own react-native projects.
Google Cloud’s Vision API offers robust pre-trained machine learning models through REST and RPC APIs. It assigns labels to images and quickly classifies them into millions of predefined classes or categories. It can detect objects and faces, read the printed or handwritten text, and build valuable metadata into your image catalog.

Step 4: —How to get google vision API key ?

Google Cloud’s Vision API offers robust pre-trained machine learning models through REST and RPC APIs. It assigns labels to images and quickly classifies them into millions of predefined classes or categories. It can detect objects and faces, read the printed or handwritten text, and build valuable metadata into your image catalog.

For implementing Google Vision in your react-native app, you can visit the given link or, follow certain steps:

1. Sign in to your Google Account.
2. Select or Create a Google Cloud Platform Project.Please make sure that billing is enabled for your Google Cloud Platform project. To enable billing, follow the procedure provided in the given link.N.B.: You will not be able to get any response from API until you enable the billing.
3. To Enable the Cloud Vision API.→ First, select a project.→ After selecting the project, click on “Go to API’s overview” button.→ Click on “Enable API and Services”,→ Search for Cloud Vision API.→ Click on the enable button.→ To use this API, click on Create Credentials.→ Choose the Cloud Vision API from the list.→ Click on “Yes I’m using one or both” and proceed further.→ Click on the Done button.→ Click on “Create Credentials”.→ Click “API key” from options.→ Copy the API key popped up and save it at a safer place. It should not be made public.→ Click the close button.

Congrats Your API key has been generated.

Now you are all set to start with google vision but before wo start coding let us integrate the most important thing for this application and that is the Camera on, so we will use react-native-camera npm package to take pictures and send those to the Google Vision API for responses,
Here I would like to inform you that the react-native-camera npm package has some bugs so you would need to install this package from the master branch.

Execute the command below to integrate camera in your application: —

yarn add react-native-camera@git+https://git@github.com/react-native-community/react-native-camera.git

You can get help from here if you get any common issues while integrating it.

Step 5: — Implementation and setup

We are now totally set to do some magic with image and google vision API , So let set go for the magic to happen…

First of all let us make some cool UI for opening the camera and showing the result.

Code View: —

Result: —

logicView: —

Result: —

Here, we have used react-native fetch method to call the API using POST method and receive the response with that.

In the code above you have “config.googleCloud.api + config.googleCloud.apiKey” which will be google cloud api and another is your api which you get after creating account and activating Google Vision Api in google console.

In the above images i have only printed webEntities responses but you can pass the features in the body and get the required responses.

Complete source code of this tutorial can be found at ReactNativeGoogleVision .

Conclusion

In this blog, you learnt how to implement Google Vision API in React Native apps for Android. You also learnt how to integrate react-native-camera in react-native application. We also built the app in Android and ran on a device.

Complete source code of this tutorial can be found at ReactNativeGoogleVision .

Next Steps

Now that you have learnt about setting up to implement Google Vision API in React Native apps for Android, here are some other topics you can look into

• Firebase — Integrate Firebase | Analytics | Push notifications | Firebase CRUD
• How To in React Native — Geolocation | Life cycle hooks | Image Picker | Redux implementation | Make API calls | Navigation | Translation | Barcode & QR code scan | Send & Read SMS | Google Vision
• Payments — Apple Pay | Stripe payments
• Authentication — Google Login| Facebook login | Phone Auth | Twitter login | Anonymous Login
• Create Instagram / Whatsapp Story Feature in React Native
• React Native life cycle hooks | Implement Redux | Async actions with Redux
• Create Awesome Apps in React Native using Full App

If you need a base to start your next React Native app, you can make your next awesome app using React Native Full App