DEV Community

Cover image for Image classification app in Android using custom TFLite model
Vishnu Sivan
Vishnu Sivan

Posted on

Image classification app in Android using custom TFLite model

Technology plays a vital role in our daily life. Artificial intelligence and machine learning are the two key players in automation. Also, image classification is the subpart of machine learning used in many areas, such as medical, education, automotive, gaming, traffic control systems and more.
TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It provides various functionalities like image classification, image segmentation, pose estimation, gesture classification, reinforcement learning and all. 
In this section, we are trying to create an image classification app in android studio using the TensorFlow lite library.

Getting Started

Image classification is a supervised learning method where we define a set of target classes and train a model to recognize them using labeled images. So, the image classification algorithm takes an image as input and classifies it into one of the output classes.
It's hard to do image classification from scratch on edge devices. The TensorFlow library is an awesome resource to build an image classification functionality in your app seamlessly.

TensorFlow examples

https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android contains an example application for TensorFlow Lite on Android. It uses Image classification to continuously classify whatever it sees from the device's back camera. The inference is performed using the TensorFlow Lite Java API. The demo app classifies frames in real-time, displaying the topmost probable classes. It allows the user to choose between a floating-point or quantized model, select the thread count, and decide whether to run on CPU, GPU, or via NNAPI.

Requirements

Android Studio 3.2+
Android SDK 28+
Java JDK 1.8+
Android device in developer mode with USB debugging enabled

Models

Tensorflow official repository (https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android) provides 4 models bundled in the app: MobileNetV1 (float), MobileNetV1 (quantized), EfficientNetLite (float) and EfficientNetLite (quantized). Particularly, we chose "mobilenet_v1_1.0_224" and "efficientnet-lite0". 
Downloading, extracting, and placing the model in the assets folder is managed automatically by download.gradle.
For details of the model used, visit Image classification.

Develop an image classification app

Tensorflow is already come up with plenty of examples. It also includes the image classification example with pre-trained generic models (which is already mentioned above). So, we are trying the pre-trained model first. After that, will try a custom model which is created using the teachable machine.

Step 1. Clone the TensorFlow examples source code

Clone the TensorFlow examples GitHub repository to your computer to get the demo application.

git clone https://github.com/tensorflow/examples
Enter fullscreen mode Exit fullscreen mode

Open the TensorFlow example in Android Studio. To do this, open Android Studio and select Open, setting the folder to examples/lite/examples/image_classification/android

android project open
android project open

Step 2. Build the Android Studio project

Select Build -> Make Project and check that the project builds successfully. 

build
This image classification app demonstrates two implementation solutions, lib_task_api that leverage the out-of-box API from the TensorFlow Lite Task Library, and lib_support that create the custom inference pipeline using the TensorFlow Lite Support Library. You can change the build variant to whichever one you want to build and run-just go to Build > Select Build Variant and select one from the drop-down menu. See configure product flavors in Android Studio for more details.
The file download.gradle directs Gradle to download the models used in the example, placing them into assets.

Step 3. Install and run the app

Connect the Android device to the computer and be sure to approve any ADB permission prompts that appear on your phone. Select Run -> Run app. Select the deployment target in the connected devices to the device on which the app will be installed. This will install the app on the device.

run
To test the app, open the app called TFL Classify on your device. When you run the app the first time, the app will request permission to access the camera. 

NB: Do not delete the assets folder content. If you explicitly deleted the files, choose Build -> Rebuild to re-download the deleted model files into the assets folder.

There you have it! Your own image classification App :)

App demo

Custom model integration

So far, we have tried image classification with pre-trained model. If you are planning to integrate a custom model into the app then we can go with the Teachable Machine online tool to train your own model. 

Step 1: Custom model creation

Open https://teachablemachine.withgoogle.com/train/image in your browser. You will get a screen like this,

Teachable machine

Provide the class name as the object name which we are trying to train here. 
After that, you can use the webcam or upload option to upload the object images. It is recommended to upload at least 20 images for an object.

Teachable machine

NB: The TensorFlow android example app code requires at least 3 classes for initializing the image classification engine. So, try to add three or more classes.

Click on the Train Model button to train your model. 
After model training the Export Model button becomes active. Click on Export Model button to get the export option window. 

Teachable machine

Select Tensorflow Lite Quantized conversion type and click on the Download my model button to download your custom model.

Teachable machine

Step 2: Custom model integration

Extract the converted_tflite_quantized.zip file to get model.tflite and labels.txt file. Create a folder named converted_tflite_quantized. Add these two files to that folder. Then copy that folder inside models\src\main\assets folder. 
Open the ClassifierQuantizedMobileNet.java file from the lib_support library and modify getLabelPath() and getModelPath() to

@Override
protected String getModelPath() {
    return "converted_tflite_quantized/model.tflite";
}

@Override
protected String getLabelPath() {
    return "converted_tflite_quantized/labels.txt";
}
Enter fullscreen mode Exit fullscreen mode

Open the ClassifierQuantizedMobileNet.java file from the lib_task_api library and modify getModelPath() to

@Override
protected String getModelPath() {
    return "converted_tflite_quantized/model.tflite";
}
Enter fullscreen mode Exit fullscreen mode

Step 3. Install and run the app

Connect the Android device to the computer and be sure to approve any ADB permission prompts that appear on your phone. Select Run -> Run app.

Run project

The labels.txt file contains labels with their indices. So, you might get only the index of the class instead of class name while running the custom model. In that case, remove the indices from the label.txt file.

There you have it! Your own image classification App with your own model :)

App demo

Thanks for reading this article.
If you enjoyed this article, please click on the heart button ♥ and share to help others find it!

If you are interested in further exploring, here are some resources I found helpful along the way:

Image Classification | Tensorflow

Image Classification Demo

Originally posted on Medium -

Image classification app in Android using custom TFLite model

Discussion (0)