DEV Community

Cover image for Hey @AWSAmplify, Is this food?
Offline Programmer
Offline Programmer

Posted on

Hey @AWSAmplify, Is this food?

#codenewbie #java #android #beginners

Scientist recommends checking before eating a "thing" if it is food. You are not supposed to eat Non Food items as it is not good for your health. Well, This is easier said than done. i.e. how on earth can you do that before you eat that "thing"?!!

Confused_Nick_Young_(wide)

I can help you with that. Today I will show you how you can use your phone and the Predictions category of @AWSAmplify to check if the "thing" is food before you eat it. We will build an App that uses machine learning to label real-world objects and confirms food items.

ill-show-you-

Amplify Setup

Open the new Amplify Admin UI click "Get Started" under "Create an app backend."

Screen Shot 2020-12-10 at 1.08.15 PM

We need to configure authentication for the App. Click on (Authentication>>) to get started.

Screen Shot 2020-12-10 at 1.08.47 PM

Click on (New app) and choose (Create app backend)

Screen Shot 2020-12-10 at 1.09.31 PM

We are going to use (IsThisAFood) as name for the App.

Screen Shot 2020-12-10 at 1.10.07 PM

Click (Confirm deployment) for Amplify to start setting up the Admin UI

Screen Shot 2020-12-10 at 1.12.31 PM

Once the setup complete, you will get the screen below

Screen Shot 2020-12-10 at 1.18.28 PM

Click on (Open admin UI)

Screen Shot 2020-12-10 at 1.19.56 PM

Click on (Enable authentication)

Screen Shot 2020-12-10 at 1.19.56 PM_dgwiu5tvm

You will find Email authentication enabled. We need to deploy by clicking on (Save and deploy)

Screen Shot 2020-12-10 at 1.20.57 PM

Wait for the deployment completion and not the pull down command

Screen Shot 2020-12-10 at 1.26.32 PM

Project Setup

Create a new Android Studio project using the Empty Activity template

Screen Shot 2020-12-10 at 2.54.33 PM

Let's call the App "Food Detector"

Screen Shot 2020-12-10 at 2.55.59 PM

Update build.gradle (Project: Food_Detector) as below

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        google()
        jcenter()
        maven { url 'https://jitpack.io' }
    }
    dependencies {
        classpath "com.android.tools.build:gradle:4.1.1"

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
        maven { url 'https://jitpack.io' }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}
Enter fullscreen mode Exit fullscreen mode

Update the dependencies section on build.gradle (Module: Food_Detector.app) as below

...

dependencies {

    // Amplify core dependency
    implementation 'com.amplifyframework:core:1.6.4'
    implementation 'com.amplifyframework:rxbindings:1.6.4'

    // Add these lines in `dependencies`
    implementation 'com.amplifyframework:aws-predictions:1.6.4'
    implementation 'com.amplifyframework:aws-auth-cognito:1.6.4'

    // Support for Java 8 features
    coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.1.1'
    implementation 'io.reactivex.rxjava2:rxjava:2.2.13'

    implementation 'com.github.esafirm.android-image-picker:imagepicker:2.3.0'
    implementation 'com.github.esafirm.android-image-picker:rximagepicker:2.3.0'
    implementation 'com.github.bumptech.glide:glide:4.11.0'
    implementation 'pub.devrel:easypermissions:3.0.0'
    implementation 'com.github.yalantis:ucrop:2.2.5'
    implementation 'com.github.bumptech.glide:glide:4.11.0'
    annotationProcessor 'com.github.bumptech.glide:compiler:4.11.0'


    implementation 'androidx.appcompat:appcompat:1.2.0'
    implementation 'com.google.android.material:material:1.2.1'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'androidx.test.ext:junit:1.1.2'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
}

...
Enter fullscreen mode Exit fullscreen mode

Pull down the Amplify backend we created above


amplify pull --appId dpcb6ka6z914 --envName staging
Enter fullscreen mode Exit fullscreen mode

Answer the prompted questions, and once complete, you will get a confirmation as below.


Scanning for plugins...
Plugin scan successful
Opening link: https://us-east-1.admin.amplifyapp.com/admin/dpcb6ka6z914/staging/verify/
✔ Successfully received Amplify Admin tokens.
Amplify AppID found: dpcb6ka6z914. Amplify App name is: IsThisAFood
Backend environment staging found in Amplify Console app: IsThisAFood
? Choose your default editor: None
? Choose the type of app that you're building android
Please tell us about your project
? Where is your Res directory:  app/src/main/res
? Do you plan on modifying this backend? Yes

Added backend environment config object to your project.
Run 'amplify pull' to sync upstream changes.
Enter fullscreen mode Exit fullscreen mode

Add Predictions by running the command below


amplify add predictions
Enter fullscreen mode Exit fullscreen mode

Use the answers below


? Please select from one of the categories below Identify
? What would you like to identify? Identify Labels
? Provide a friendly name for your resource identifyLabels241c2197
? Would you like use the default configuration? Default Configuration
? Who should have access? Auth and Guest users
Enter fullscreen mode Exit fullscreen mode

Once complete you will get the confirmation below


Successfully updated auth resource locally.
Successfully added resource identifyLabels241c2197 locally
Enter fullscreen mode Exit fullscreen mode

Publish those changes by running the command below


amplify push
Enter fullscreen mode Exit fullscreen mode

Create a new class and name it (FoodDetector). We will use this class to initialize Amplify as below - note how the class extends from Application


public class FoodDetector extends Application {
    @Override
    public void onCreate() {
        super.onCreate();

        try {
            // Add these lines to add the AWSCognitoAuthPlugin and AWSPredictionsPlugin plugins
            RxAmplify.addPlugin(new AWSCognitoAuthPlugin());
            RxAmplify.addPlugin(new AWSPredictionsPlugin());
            RxAmplify.configure(getApplicationContext());

            Log.i("MyAmplifyApp", "Initialized Amplify");
        } catch (AmplifyException error) {
            Log.e("MyAmplifyApp", "Could not initialize Amplify", error);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Update AndroidManifest.xml to add a android:name attribute with the value of .FoodDetector as below


...

<application
        android:name=".FoodDetector"

...
Enter fullscreen mode Exit fullscreen mode

The UI

Create the two vector assets shown below

Screen Shot 2020-12-13 at 7.31.24 PM

Screen Shot 2020-12-13 at 7.34.02 PM

Update activity_main.xml as below


<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <androidx.constraintlayout.widget.Guideline
        android:id="@+id/guideline"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:orientation="horizontal"
        app:layout_constraintGuide_percent="0.6" />

    <FrameLayout
        android:id="@+id/photo_FrameLayout"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_marginTop="15dp"
        android:paddingStart="20dp"
        android:paddingEnd="20dp"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent">

        <ImageView
            android:id="@+id/capturedImage"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:scaleType="fitCenter"
            android:adjustViewBounds="true"
            app:layout_constraintTop_toTopOf="parent"
            app:layout_constraintStart_toStartOf="parent"
            app:layout_constraintBottom_toTopOf="@id/guideline"
            app:layout_constraintEnd_toEndOf="parent" />

            <ImageView
                android:id="@+id/flagImage"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:scaleType="fitCenter"
                android:adjustViewBounds="true"
                android:visibility="gone"
                android:src="@drawable/ic_baseline_close_24" />

    </FrameLayout>

    <Button
        android:id="@+id/capture_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginEnd="8dp"
        android:background="@color/colorPrimary"
        android:padding="12dp"
        android:text="Select Photo"
        app:layout_constraintTop_toBottomOf="@id/guideline"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintBottom_toTopOf="@id/objectTextView"
        app:layout_constraintEnd_toEndOf="parent"
        />

    <TextView
        android:textSize="20sp"
        android:id="@+id/objectTextView"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="I am hungry!!!"
        android:inputType="textMultiLine"
        android:layout_margin="20dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toBottomOf="@id/capture_button" />

</androidx.constraintlayout.widget.ConstraintLayout>
Enter fullscreen mode Exit fullscreen mode

Photo Selection

The user will capture or select a photo to check if it is food. We are going to use Glide and UCrop libraries for that.

Create a new class (MyAppGlideModule) as below


import com.bumptech.glide.annotation.GlideModule;
import com.bumptech.glide.module.AppGlideModule;

// new since Glide v4
@GlideModule
public final class MyAppGlideModule extends AppGlideModule {
    // leave empty for now
}
Enter fullscreen mode Exit fullscreen mode

Update themes.xml to set a NoActionBar theme


...

    <style name="AppTheme.NoActionBar" parent="Theme.MaterialComponents.Light.NoActionBar">
        <item name="windowActionBar">false</item>
        <item name="windowNoTitle">true</item>
        <item name="colorPrimary">@color/colorPrimary</item>
        <item name="colorPrimaryDark">@color/colorPrimaryDark</item>
        <item name="colorAccent">@color/colorAccent</item>

        <item name="android:navigationBarColor" tools:targetApi="lollipop">@android:color/black
        </item>
    </style>

...
Enter fullscreen mode Exit fullscreen mode

Update AndroidManifest.xml to set the required permissions and the UCropActivity


<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.offlineprogrammer.fooddetector">

    <uses-feature android:name="android.hardware.camera" android:required="true" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.INTERNET" />

    <application
        android:requestLegacyExternalStorage="true"
        android:name=".FoodDetector"
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/Theme.FoodDetector">
        <activity
            android:name="com.yalantis.ucrop.UCropActivity"
            android:screenOrientation="portrait"
            android:theme="@style/Theme.AppCompat.Light.NoActionBar" />
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>
Enter fullscreen mode Exit fullscreen mode

Update MainActivity.java to identify the UI components

...

Button capture_button;
ImageView capturedImage;
TextView objectTextView;
ImageView flagImage;
ProgressDialog progressDialog;

...

capturedImage = findViewById(R.id.capturedImage);
objectTextView = findViewById(R.id.objectTextView);
capture_button = findViewById(R.id.capture_button);
flagImage = findViewById(R.id.flagImage);

...
Enter fullscreen mode Exit fullscreen mode

Set the OnClick listener for the capture_button as below

...

capture_button.setOnClickListener(view -> {
            ImagePicker.create(MainActivity.this).returnMode(ReturnMode.ALL)
                    .folderMode(true).includeVideo(false).limit(1).theme(R.style.AppTheme_NoActionBar).single().start();
        });

...
Enter fullscreen mode Exit fullscreen mode

Add the methods below to get the required permissions for using the Camera

...

    public void onRequestPermissionsResult(int i, @NonNull String[] strArr, @NonNull int[] iArr) {
        super.onRequestPermissionsResult(i, strArr, iArr);
        EasyPermissions.onRequestPermissionsResult(i, strArr, iArr, this);
    }


    public void onPermissionsGranted(int i, @NonNull List<String> list) {
        Intent intent = new Intent("android.media.action.IMAGE_CAPTURE");
        startActivityForResult(intent, CAMERA_REQUEST);
    }

    public void onPermissionsDenied(int i, @NonNull List<String> list) {
        if (EasyPermissions.somePermissionPermanentlyDenied(this, list)) {
            new AppSettingsDialog.Builder(this).setTitle("Permissions Required").setPositiveButton("Settings").setNegativeButton("Cancel").setRequestCode(5).build().show();
        }
    }

...
Enter fullscreen mode Exit fullscreen mode

Add the methods below to display the selected\captured image

...

    public void onActivityResult(int i, int i2, Intent intent) {
        super.onActivityResult(i, i2, intent);
        if (ImagePicker.shouldHandle(i, i2, intent)) {
            Image firstImageOrNull = ImagePicker.getFirstImageOrNull(intent);
            if (firstImageOrNull != null) {
                UCrop.of(Uri.fromFile(new File(firstImageOrNull.getPath())), Uri.fromFile(new File(getCacheDir(), "cropped"))).withAspectRatio(1.0f, 1.0f).start(this);
            }
        }
        if (i == UCrop.REQUEST_CROP) {
            onCropFinish(intent);
        }
    }

    public void onCropFinish(Intent intent) {
        if (intent == null) {
            return;
        }
        progressDialog = new ProgressDialog(this);
        progressDialog.setTitle("Processing...");
        progressDialog.show();
        GlideApp.with(this)
                .asBitmap()
                .load(UCrop.getOutput(intent).getPath()).diskCacheStrategy(DiskCacheStrategy.NONE).skipMemoryCache(true).centerCrop()
                .into(new CustomTarget<Bitmap>() {
                    @Override
                    public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
                        capturedImage.setImageBitmap(resource);
                    }
                    @Override
                    public void onLoadCleared(@Nullable Drawable placeholder) {
                    }
                });
    }

...
Enter fullscreen mode Exit fullscreen mode

Food Detection

Add the method below to detect the label and update the flag image

...

    public void detectLabels(Bitmap image) {
        RxAmplify.Predictions.identify(LabelType.LABELS, image)
                .subscribe(
                        result -> {
                            String sLabel = "";
                            IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
                            //    Label label = identifyResult.getLabels().get(0);
                            List<Label> labels = identifyResult.getLabels();
                            for (Label label : labels) {
                                sLabel += String.format(" %s | ",label.getName());
                                if("food".equalsIgnoreCase(label.getName())){
                                    flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_check_24));
                                }
                            }
                            String finalSLabel = sLabel;
                            runOnUiThread(new Runnable() {
                                @Override
                                public void run() {
                                    objectTextView.setText(finalSLabel);
                                    flagImage.setVisibility(View.VISIBLE);
                                    progressDialog.dismiss();

                                }
                            });
                            Log.i("MyAmplifyApp", sLabel);
                        },
                        error -> Log.e("MyAmplifyApp", "Label detection failed", error)
                );
    }
    ...
Enter fullscreen mode Exit fullscreen mode

We will call the detectLabels method when the App displays the selected\captured photo as below

...

    @Override
    public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
        capturedImage.setImageBitmap(resource);
        detectLabels(resource);
    }

...
Enter fullscreen mode Exit fullscreen mode

Now MainActivity.java would look like below

...

public class MainActivity extends AppCompatActivity {

    Button capture_button;
    ImageView capturedImage;
    TextView objectTextView;
    ImageView flagImage;
    ProgressDialog progressDialog;
    private static final int CAMERA_REQUEST = 2222;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        capturedImage = findViewById(R.id.capturedImage);
        objectTextView = findViewById(R.id.objectTextView);
        capture_button = findViewById(R.id.capture_button);
        flagImage = findViewById(R.id.flagImage);

        capture_button.setOnClickListener(view -> {
            ImagePicker.create(MainActivity.this).returnMode(ReturnMode.ALL)
                    .folderMode(true).includeVideo(false).limit(1).theme(R.style.AppTheme_NoActionBar).single().start();
            flagImage.setVisibility(View.GONE);
            flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_close_24));

        });

    }

    public void onRequestPermissionsResult(int i, @NonNull String[] strArr, @NonNull int[] iArr) {
        super.onRequestPermissionsResult(i, strArr, iArr);
        EasyPermissions.onRequestPermissionsResult(i, strArr, iArr, this);
    }


    public void onPermissionsGranted(int i, @NonNull List<String> list) {
        Intent intent = new Intent("android.media.action.IMAGE_CAPTURE");
        startActivityForResult(intent, CAMERA_REQUEST);
    }

    public void onPermissionsDenied(int i, @NonNull List<String> list) {
        if (EasyPermissions.somePermissionPermanentlyDenied(this, list)) {
            new AppSettingsDialog.Builder(this).setTitle("Permissions Required").setPositiveButton("Settings").setNegativeButton("Cancel").setRequestCode(5).build().show();
        }
    }

    public void onActivityResult(int i, int i2, Intent intent) {
        super.onActivityResult(i, i2, intent);
        if (ImagePicker.shouldHandle(i, i2, intent)) {
            Image firstImageOrNull = ImagePicker.getFirstImageOrNull(intent);
            if (firstImageOrNull != null) {
                UCrop.of(Uri.fromFile(new File(firstImageOrNull.getPath())), Uri.fromFile(new File(getCacheDir(), "cropped"))).withAspectRatio(1.0f, 1.0f).start(this);
            }
        }
        if (i == UCrop.REQUEST_CROP) {
            onCropFinish(intent);
        }
    }

    public void onCropFinish(Intent intent) {
        if (intent == null) {
            return;
        }
        progressDialog = new ProgressDialog(this);
        progressDialog.setTitle("Processing...");
        progressDialog.show();

        GlideApp.with(this)
                .asBitmap()
                .load(UCrop.getOutput(intent).getPath()).diskCacheStrategy(DiskCacheStrategy.NONE).skipMemoryCache(true).centerCrop()
                .into(new CustomTarget<Bitmap>() {
                    @Override
                    public void onResourceReady(@NonNull Bitmap resource, @Nullable Transition<? super Bitmap> transition) {
                        capturedImage.setImageBitmap(resource);
                        detectLabels(resource);
                    }
                    @Override
                    public void onLoadCleared(@Nullable Drawable placeholder) {
                    }
                });
    }

    public void detectLabels(Bitmap image) {
        RxAmplify.Predictions.identify(LabelType.LABELS, image)
                .subscribe(
                        result -> {
                            String sLabel = "";
                            IdentifyLabelsResult identifyResult = (IdentifyLabelsResult) result;
                            //    Label label = identifyResult.getLabels().get(0);
                            List<Label> labels = identifyResult.getLabels();
                            for (Label label : labels) {
                                sLabel += String.format(" %s | ",label.getName());
                                if("food".equalsIgnoreCase(label.getName())){
                                    flagImage.setImageDrawable(getDrawable(R.drawable.ic_baseline_check_24));
                                }
                            }
                            String finalSLabel = sLabel;
                            runOnUiThread(new Runnable() {
                                @Override
                                public void run() {
                                    objectTextView.setText(finalSLabel);
                                    flagImage.setVisibility(View.VISIBLE);
                                    progressDialog.dismiss();

                                }
                            });
                            Log.i("MyAmplifyApp", sLabel);
                        },
                        error -> Log.e("MyAmplifyApp", "Label detection failed", error)
                );
    }
}

...
Enter fullscreen mode Exit fullscreen mode

Run the App

Check the code here

Follow me on Twitter for more tips about #coding, #learning, #technology, #Java, #JavaScript, #Autism, #Parenting...etc.

Check my Apps on Google Play

Cover image Dan Gold on Unsplash

Top comments (2)

Collapse
 
aspittel profile image
Ali Spittel •

This is awesome! What a cool app!

Collapse
 
offlineprogrammer profile image
Offline Programmer •

Thanks

Read next

bekbrace profile image

Build TUI application with GO (tview)

Bek Brace -

iqra_firdose profile image

A Beginner’s Guide to Computer Networks and the OSI Model

Iqra Firdose -

bst53 profile image

JavaScript Higher-Order Functions Made Easy: Learn with a Real-Life Example! 💡

Burhanuddin S. Tinwala -

msnmongare profile image

How to Download and Install FortiClient VPN for Your Windows PC

Sospeter Mong'are -