Technology plays a vital role in our daily life. Artificial intelligence and machine learning are the two key players in automation. Also, image classification is the subpart of machine learning used in many areas, such as medical, education, automotive, gaming, traffic control systems and more.
TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It provides various functionalities like image classification, image segmentation, pose estimation, gesture classification, reinforcement learning and all.
In this section, we are trying to create an image classification app in android studio using the TensorFlow lite library.
Getting Started
Image classification is a supervised learning method where we define a set of target classes and train a model to recognize them using labeled images. So, the image classification algorithm takes an image as input and classifies it into one of the output classes.
It's hard to do image classification from scratch on edge devices. The TensorFlow library is an awesome resource to build an image classification functionality in your app seamlessly.
https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android contains an example application for TensorFlow Lite on Android. It uses Image classification to continuously classify whatever it sees from the device's back camera. The inference is performed using the TensorFlow Lite Java API. The demo app classifies frames in real-time, displaying the topmost probable classes. It allows the user to choose between a floating-point or quantized model, select the thread count, and decide whether to run on CPU, GPU, or via NNAPI.
Requirements
Android Studio 3.2+
Android SDK 28+
Java JDK 1.8+
Android device in developer mode with USB debugging enabled
Models
Tensorflow official repository (https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android) provides 4 models bundled in the app: MobileNetV1 (float), MobileNetV1 (quantized), EfficientNetLite (float) and EfficientNetLite (quantized). Particularly, we chose "mobilenet_v1_1.0_224" and "efficientnet-lite0".
Downloading, extracting, and placing the model in the assets folder is managed automatically by download.gradle.
For details of the model used, visit Image classification.
Develop an image classification app
Tensorflow is already come up with plenty of examples. It also includes the image classification example with pre-trained generic models (which is already mentioned above). So, we are trying the pre-trained model first. After that, will try a custom model which is created using the teachable machine.
Step 1. Clone the TensorFlow examples source code
Clone the TensorFlow examples GitHub repository to your computer to get the demo application.
git clone https://github.com/tensorflow/examples
Open the TensorFlow example in Android Studio. To do this, open Android Studio and select Open, setting the folder to examples/lite/examples/image_classification/android
Step 2. Build the Android Studio project
Select Build -> Make Project and check that the project builds successfully.
This image classification app demonstrates two implementation solutions, lib_task_api that leverage the out-of-box API from the TensorFlow Lite Task Library, and lib_support that create the custom inference pipeline using the TensorFlow Lite Support Library. You can change the build variant to whichever one you want to build and run-just go to Build > Select Build Variant and select one from the drop-down menu. See configure product flavors in Android Studio for more details.
The file download.gradle directs Gradle to download the models used in the example, placing them into assets.
Step 3. Install and run the app
Connect the Android device to the computer and be sure to approve any ADB permission prompts that appear on your phone. Select Run -> Run app. Select the deployment target in the connected devices to the device on which the app will be installed. This will install the app on the device.
To test the app, open the app called TFL Classify on your device. When you run the app the first time, the app will request permission to access the camera.
NB: Do not delete the assets folder content. If you explicitly deleted the files, choose Build -> Rebuild to re-download the deleted model files into the assets folder.
There you have it! Your own image classification App :)
Custom model integration
So far, we have tried image classification with pre-trained model. If you are planning to integrate a custom model into the app then we can go with the Teachable Machine online tool to train your own model.
Step 1: Custom model creation
Open https://teachablemachine.withgoogle.com/train/image in your browser. You will get a screen like this,
Provide the class name as the object name which we are trying to train here.
After that, you can use the webcam or upload option to upload the object images. It is recommended to upload at least 20 images for an object.
NB: The TensorFlow android example app code requires at least 3 classes for initializing the image classification engine. So, try to add three or more classes.
Click on the Train Model button to train your model.
After model training the Export Model button becomes active. Click on Export Model button to get the export option window.
Select Tensorflow Lite Quantized conversion type and click on the Download my model button to download your custom model.
Step 2: Custom model integration
Extract the converted_tflite_quantized.zip file to get model.tflite and labels.txt file. Create a folder named converted_tflite_quantized. Add these two files to that folder. Then copy that folder inside models\src\main\assets folder.
Open the ClassifierQuantizedMobileNet.java file from the lib_support library and modify getLabelPath() and getModelPath() to
@Override
protected String getModelPath() {
return "converted_tflite_quantized/model.tflite";
}
@Override
protected String getLabelPath() {
return "converted_tflite_quantized/labels.txt";
}
Open the ClassifierQuantizedMobileNet.java file from the lib_task_api library and modify getModelPath() to
@Override
protected String getModelPath() {
return "converted_tflite_quantized/model.tflite";
}
Step 3. Install and run the app
Connect the Android device to the computer and be sure to approve any ADB permission prompts that appear on your phone. Select Run -> Run app.
The labels.txt file contains labels with their indices. So, you might get only the index of the class instead of class name while running the custom model. In that case, remove the indices from the label.txt file.
There you have it! Your own image classification App with your own model :)
Thanks for reading this article.
If you enjoyed this article, please click on the heart button ♥ and share to help others find it!
If you are interested in further exploring, here are some resources I found helpful along the way:
Image Classification | Tensorflow
Originally posted on Medium -
Image classification app in Android using custom TFLite model
Top comments (0)