Scientist recommends checking before eating a "thing" if it is food. You are not supposed to eat Non Food items as it is not good for your health. Well, This is easier said than done. i.e. how on earth can you do that before you eat that "thing"?!!
I can help you with that. Today I will show you how you can use your phone and the Predictions category of @AWSAmplify to check if the "thing" is food before you eat it. We will build an App that uses machine learning to label real-world objects and confirms food items.
Amplify Setup
Open the new Amplify Admin UI click "Get Started" under "Create an app backend."
We need to configure authentication for the App. Click on (Authentication>>) to get started.
Click on (New app) and choose (Create app backend)
We are going to use (IsThisAFood) as name for the App.
Click (Confirm deployment) for Amplify to start setting up the Admin UI
Once the setup complete, you will get the screen below
Click on (Open admin UI)
Click on (Enable authentication)
You will find Email authentication enabled. We need to deploy by clicking on (Save and deploy)
Wait for the deployment completion and not the pull down command
Project Setup
Create a new Android Studio project using the Empty Activity template
Let's call the App "Food Detector"
Update build.gradle (Project: Food_Detector) as below
// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
repositories {
google()
jcenter()
maven { url 'https://jitpack.io' }
}
dependencies {
classpath "com.android.tools.build:gradle:4.1.1"
// NOTE: Do not place your application dependencies here; they belong
// in the individual module build.gradle files
}
}
allprojects {
repositories {
google()
jcenter()
maven { url 'https://jitpack.io' }
}
}
task clean(type: Delete) {
delete rootProject.buildDir
}
Update the dependencies section on build.gradle (Module: Food_Detector.app) as below
...
dependencies {
// Amplify core dependency
implementation 'com.amplifyframework:core:1.6.4'
implementation 'com.amplifyframework:rxbindings:1.6.4'
// Add these lines in `dependencies`
implementation 'com.amplifyframework:aws-predictions:1.6.4'
implementation 'com.amplifyframework:aws-auth-cognito:1.6.4'
// Support for Java 8 features
coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.1.1'
implementation 'io.reactivex.rxjava2:rxjava:2.2.13'
implementation 'com.github.esafirm.android-image-picker:imagepicker:2.3.0'
implementation 'com.github.esafirm.android-image-picker:rximagepicker:2.3.0'
implementation 'com.github.bumptech.glide:glide:4.11.0'
implementation 'pub.devrel:easypermissions:3.0.0'
implementation 'com.github.yalantis:ucrop:2.2.5'
implementation 'com.github.bumptech.glide:glide:4.11.0'
annotationProcessor 'com.github.bumptech.glide:compiler:4.11.0'
implementation 'androidx.appcompat:appcompat:1.2.0'
implementation 'com.google.android.material:material:1.2.1'
implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
testImplementation 'junit:junit:4.+'
androidTestImplementation 'androidx.test.ext:junit:1.1.2'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
}
...
Pull down the Amplify backend we created above
amplify pull --appId dpcb6ka6z914 --envName staging
Answer the prompted questions, and once complete, you will get a confirmation as below.
Scanning for plugins...
Plugin scan successful
Opening link: https://us-east-1.admin.amplifyapp.com/admin/dpcb6ka6z914/staging/verify/
✔ Successfully received Amplify Admin tokens.
Amplify AppID found: dpcb6ka6z914. Amplify App name is: IsThisAFood
Backend environment staging found in Amplify Console app: IsThisAFood
? Choose your default editor: None
? Choose the type of app that you're building android
Please tell us about your project
? Where is your Res directory: app/src/main/res
? Do you plan on modifying this backend? Yes
Added backend environment config object to your project.
Run 'amplify pull' to sync upstream changes.
Add Predictions by running the command below
amplify add predictions
Use the answers below
? Please select from one of the categories below Identify
? What would you like to identify? Identify Labels
? Provide a friendly name for your resource identifyLabels241c2197
? Would you like use the default configuration? Default Configuration
? Who should have access? Auth and Guest users
Once complete you will get the confirmation below
Successfully updated auth resource locally.
Successfully added resource identifyLabels241c2197 locally
Publish those changes by running the command below
amplify push
Create a new class and name it (FoodDetector). We will use this class to initialize Amplify as below - note how the class extends from Application
public class FoodDetector extends Application {
@Override
public void onCreate() {
super.onCreate();
try {
// Add these lines to add the AWSCognitoAuthPlugin and AWSPredictionsPlugin plugins
RxAmplify.addPlugin(new AWSCognitoAuthPlugin());
RxAmplify.addPlugin(new AWSPredictionsPlugin());
RxAmplify.configure(getApplicationContext());
Log.i("MyAmplifyApp", "Initialized Amplify");
} catch (AmplifyException error) {
Log.e("MyAmplifyApp", "Could not initialize Amplify", error);
}
}
}
Update AndroidManifest.xml to add a android:name attribute with the value of .FoodDetector as below
...
<application
android:name=".FoodDetector"
...
The UI
Create the two vector assets shown below
Update activity_main.xml as below
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<androidx.constraintlayout.widget.Guideline
android:id="@+id/guideline"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:orientation="horizontal"
app:layout_constraintGuide_percent="0.6" />
<FrameLayout
android:id="@+id/photo_FrameLayout"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="15dp"
android:paddingStart="20dp"
android:paddingEnd="20dp"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent">
<ImageView
android:id="@+id/capturedImage"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:scaleType="fitCenter"
android:adjustViewBounds="true"
app:layout_constraintTop_toTopOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintBottom_toTopOf="@id/guideline"
app:layout_constraintEnd_toEndOf="parent" />
<ImageView
android:id="@+id/flagImage"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:scaleType="fitCenter"
android:adjustViewBounds="true"
android:visibility="gone"
android:src="@drawable/ic_baseline_close_24" />
</FrameLayout>
<Button
android:id="@+id/capture_button"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginEnd="8dp"
android:background="@color/colorPrimary"
android:padding="12dp"
android:text="Select Photo"
app:layout_constraintTop_toBottomOf="@id/guideline"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintBottom_toTopOf="@id/objectTextView"
app:layout_constraintEnd_toEndOf="parent"
/>
<TextView
android:textSize="20sp"
android:id="@+id/objectTextView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="I am hungry!!!"
android:inputType="textMultiLine"
android:layout_margin="20dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toBottomOf="@id/capture_button" />
</androidx.constraintlayout.widget.ConstraintLayout>
Photo Selection
The user will capture or select a photo to check if it is food. We are going to use Glide and UCrop libraries for that.
Create a new class (MyAppGlideModule) as below
import com.bumptech.glide.annotation.GlideModule;
import com.bumptech.glide.module.AppGlideModule;
// new since Glide v4
@GlideModule
public final class MyAppGlideModule extends AppGlideModule {
// leave empty for now
}
Update themes.xml to set a NoActionBar theme
...
<style name="AppTheme.NoActionBar" parent="Theme.MaterialComponents.Light.NoActionBar">
<item name="windowActionBar">false</item>
<item name="windowNoTitle">true</item>
<item name="colorPrimary">@color/colorPrimary</item>
<item name="colorPrimaryDark">@color/colorPrimaryDark</item>
<item name="colorAccent">@color/colorAccent</item>
<item name="android:navigationBarColor" tools:targetApi="lollipop">@android:color/black
</item>
</style>
...
Update AndroidManifest.xml to set the required permissions and the UCropActivity
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.offlineprogrammer.fooddetector">
<uses-feature android:name="android.hardware.camera" android:required="true" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.INTERNET" />
<application
android:requestLegacyExternalStorage="true"
android:name=".FoodDetector"
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:roundIcon="@mipmap/ic_launcher_round"
android:supportsRtl="true"
android:theme="@style/Theme.FoodDetector">
<activity
android:name="com.yalantis.ucrop.UCropActivity"
android:screenOrientation="portrait"
android:theme="@style/Theme.AppCompat.Light.NoActionBar" />
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>
Update MainActivity.java to identify the UI components
...
Button capture_button;
ImageView capturedImage;
TextView objectTextView;
ImageView flagImage;
ProgressDialog progressDialog;
...
capturedImage = findViewById(R.id.capturedImage);
objectTextView = findViewById(R.id.objectTextView);
capture_button = findViewById(R.id.capture_button);
flagImage = findViewById(R.id.flagImage);
...
Set the OnClick listener for the capture_button as below
...
capture_button.setOnClickListener(view -> {
ImagePicker.create(MainActivity.this).returnMode(ReturnMode.ALL)
.folderMode(true).includeVideo(false).limit(1).theme(R.style.AppTheme_NoActionBar).single().start();
});
...
Add the methods below to get the required permissions for using the Camera
...
public void onRequestPermissionsResult(int i, @NonNull String[] strArr, @NonNull int[] iArr) {
super.onRequestPermissionsResult(i, strArr, iArr);
EasyPermissions.onRequestPermissionsResult(i, strArr, iArr, this);
}
public void onPermissionsGranted(int i, @NonNull List<String> list) {
Intent intent = new Intent("android.media.action.IMAGE_CAPTURE");
startActivityForResult(intent, CAMERA_REQUEST);
}
public void onPermissionsDenied(int i, @NonNull List<String> list) {
if (EasyPermissions.somePermissionPermanentlyDenied(this, list)) {
new AppSettingsDialog.Builder(this).setTitle("Permissions Required").setPositiveButton("Settings").setNegativeButton("Cancel").setRequestCode(5).build().show();
}
}
...
Add the methods below to display the selected\captured image
...
public void onActivityResult(int i, int i2, Intent intent) {
super.onActivityResult(i, i2, intent);
if (ImagePicker.shouldHandle(i, i2, intent)) {
Image firstImageOrNull = ImagePicker.getFirstImageOrNull(intent);
if (firstImageOrNull != null) {
UCrop.of(Uri.fromFile(new File(firstImageOrNull.getPath())), Uri.fromFile(new File(getCacheDir(), "cropped"))).withAspectRatio(1.0f, 1.0f).start(this);
}
}
if (i == UCrop.REQUEST_CROP) {
onCropFinish(intent);
}
}
public void onCropFinish(Intent intent) {
if (intent == null) {
return;
}
progressDialog = new ProgressDialog(this);
progressDialog.setTitle("Processing...");
progressDialog.show();
GlideApp.with(this)
.asBitmap()
.load(UCrop.getOutput(intent).getPath()).diskCacheStrategy(DiskCacheStrategy.NONE).skipMemoryCache(true).centerCrop()
.into(new CustomTarget<Bitmap>() {
@Override
public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
capturedImage.setImageBitmap(resource);
}
@Override
public void onLoadCleared(@Nullable Drawable placeholder) {
}
});
}
...
Food Detection
Add the method below to detect the label and update the flag image
...
public void detectLabels(Bitmap image) {
RxAmplify.Predictions.identify(LabelType.LABELS, image)
.subscribe(
result -> {
String sLabel = "";
IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
// Label label = identifyResult.getLabels().get(0);
List<Label> labels = identifyResult.getLabels();
for (Label label : labels) {
sLabel += String.format(" %s | ",label.getName());
if("food".equalsIgnoreCase(label.getName())){
flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_check_24));
}
}
String finalSLabel = sLabel;
runOnUiThread(new Runnable() {
@Override
public void run() {
objectTextView.setText(finalSLabel);
flagImage.setVisibility(View.VISIBLE);
progressDialog.dismiss();
}
});
Log.i("MyAmplifyApp", sLabel);
},
error -> Log.e("MyAmplifyApp", "Label detection failed", error)
);
}
...
We will call the detectLabels method when the App displays the selected\captured photo as below
...
@Override
public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
capturedImage.setImageBitmap(resource);
detectLabels(resource);
}
...
Now MainActivity.java would look like below
...
public class MainActivity extends AppCompatActivity {
Button capture_button;
ImageView capturedImage;
TextView objectTextView;
ImageView flagImage;
ProgressDialog progressDialog;
private static final int CAMERA_REQUEST = 2222;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
capturedImage = findViewById(R.id.capturedImage);
objectTextView = findViewById(R.id.objectTextView);
capture_button = findViewById(R.id.capture_button);
flagImage = findViewById(R.id.flagImage);
capture_button.setOnClickListener(view -> {
ImagePicker.create(MainActivity.this).returnMode(ReturnMode.ALL)
.folderMode(true).includeVideo(false).limit(1).theme(R.style.AppTheme_NoActionBar).single().start();
flagImage.setVisibility(View.GONE);
flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_close_24));
});
}
public void onRequestPermissionsResult(int i, @NonNull String[] strArr, @NonNull int[] iArr) {
super.onRequestPermissionsResult(i, strArr, iArr);
EasyPermissions.onRequestPermissionsResult(i, strArr, iArr, this);
}
public void onPermissionsGranted(int i, @NonNull List<String> list) {
Intent intent = new Intent("android.media.action.IMAGE_CAPTURE");
startActivityForResult(intent, CAMERA_REQUEST);
}
public void onPermissionsDenied(int i, @NonNull List<String> list) {
if (EasyPermissions.somePermissionPermanentlyDenied(this, list)) {
new AppSettingsDialog.Builder(this).setTitle("Permissions Required").setPositiveButton("Settings").setNegativeButton("Cancel").setRequestCode(5).build().show();
}
}
public void onActivityResult(int i, int i2, Intent intent) {
super.onActivityResult(i, i2, intent);
if (ImagePicker.shouldHandle(i, i2, intent)) {
Image firstImageOrNull = ImagePicker.getFirstImageOrNull(intent);
if (firstImageOrNull != null) {
UCrop.of(Uri.fromFile(new File(firstImageOrNull.getPath())), Uri.fromFile(new File(getCacheDir(), "cropped"))).withAspectRatio(1.0f, 1.0f).start(this);
}
}
if (i == UCrop.REQUEST_CROP) {
onCropFinish(intent);
}
}
public void onCropFinish(Intent intent) {
if (intent == null) {
return;
}
progressDialog = new ProgressDialog(this);
progressDialog.setTitle("Processing...");
progressDialog.show();
GlideApp.with(this)
.asBitmap()
.load(UCrop.getOutput(intent).getPath()).diskCacheStrategy(DiskCacheStrategy.NONE).skipMemoryCache(true).centerCrop()
.into(new CustomTarget<Bitmap>() {
@Override
public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
capturedImage.setImageBitmap(resource);
detectLabels(resource);
}
@Override
public void onLoadCleared(@Nullable Drawable placeholder) {
}
});
}
public void detectLabels(Bitmap image) {
RxAmplify.Predictions.identify(LabelType.LABELS, image)
.subscribe(
result -> {
String sLabel = "";
IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
// Label label = identifyResult.getLabels().get(0);
List<Label> labels = identifyResult.getLabels();
for (Label label : labels) {
sLabel += String.format(" %s | ",label.getName());
if("food".equalsIgnoreCase(label.getName())){
flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_check_24));
}
}
String finalSLabel = sLabel;
runOnUiThread(new Runnable() {
@Override
public void run() {
objectTextView.setText(finalSLabel);
flagImage.setVisibility(View.VISIBLE);
progressDialog.dismiss();
}
});
Log.i("MyAmplifyApp", sLabel);
},
error -> Log.e("MyAmplifyApp", "Label detection failed", error)
);
}
}
...
Run the App
Check the code here
Follow me on Twitter for more tips about #coding, #learning, #technology, #Java, #JavaScript, #Autism, #Parenting...etc.
Check my Apps on Google Play
Oldest comments (2)
This is awesome! What a cool app!
Thanks