DEV Community

Cover image for Building an Image Recognition Website with SvelteKit and TensorFlow.js
Estelle-K
Estelle-K

Posted on

Building an Image Recognition Website with SvelteKit and TensorFlow.js

#beginners #tutorial #machinelearning #svelte

Introduction

In this article, I'll show you how to build a simple website that allows users to upload an image and get predictions about its content using MobileNet model from TensorFlow.js. This project is perfect for beginners in TensorFlow.js who want to get started with machine learning in the browser.

Prerequisites

Before you start, you should have some knowledge of web development and be familiar with JavaScript, TypeScript and Tailwind CSS. Make sure the following tools are also installed:

Step 1: Create a new SvelteKit project

Start by creating a new SvelteKit project using the following command in the terminal:

npm init svelte@next my-project
Enter fullscreen mode Exit fullscreen mode

Screenshot of SvelteKit installation

Screenshot of the end of SvelteKit installation

Step 2: Install Tailwind CSS and Flowbite Svelte

Install Tailwind CSS in a SvelteKit project in the terminal:

npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p
Enter fullscreen mode Exit fullscreen mode

Update your tailwind.config.js file:

/** @type {import('tailwindcss').Config} */
export default {
  content: ['./src/**/*.{html,js,svelte,ts}'],
  theme: {
    extend: {}
  },
  plugins: []
};
Enter fullscreen mode Exit fullscreen mode

Create ./src/app.css file:

@tailwind base;
@tailwind components;
@tailwind utilities;
Enter fullscreen mode Exit fullscreen mode

Create a ./src/routes/+layout.sveltefle and importe the app.css file:

<script>
  import "../app.css";
</script>

<slot />
Enter fullscreen mode Exit fullscreen mode

Install Flowbite Svelte in the terminal:

npm i -D flowbite-svelte flowbite
Enter fullscreen mode Exit fullscreen mode

Update your tailwind.config.js file:

/** @type {import('tailwindcss').Config} */
export default {
    content: [
        './src/**/*.{html,js,svelte,ts}',
        './node_modules/flowbite-svelte/**/*.{html,js,svelte,ts}'
    ],
    theme: {
        extend: {
            colors: {
                // flowbite-svelte
                primary: {
                    50: '#FFF5F2',
                    100: '#FFF1EE',
                    200: '#FFE4DE',
                    300: '#FFD5CC',
                    400: '#FFBCAD',
                    500: '#FE795D',
                    600: '#EF562F',
                    700: '#EB4F27',
                    800: '#CC4522',
                    900: '#A5371B'
                }
            }
        }
    },
    plugins: [require('flowbite/plugin')]
};
Enter fullscreen mode Exit fullscreen mode

Step 3: Install TensorFlow.js

Install TensorFlow.js in the terminal:

npm i @tensorflow-models/mobilenet @tensorflow/tfjs @tensorflow/tfjs-vis
Enter fullscreen mode Exit fullscreen mode

Step 4: Import the MobileNet model

Go to ./src/routes/+page.svelte and delete everything there.

Create a <script lang="ts"></script> tag and add these imports:

<script lang="ts">
    import * as tf from '@tensorflow/tfjs';
    import * as mobilenet from '@tensorflow-models/mobilenet';
</script>
Enter fullscreen mode Exit fullscreen mode

Step 5: Create a function to load the model

Create a function to load the MobileNet model and return it. Add this code in script tag under imports:

let model: mobilenet.MobileNet;

const loadModel = async () => {
        const version = 2;
        const alpha = 0.5;
        model = await mobilenet.load({ version, alpha });
        console.log('Model loaded successfully');
        return model;
    };
Enter fullscreen mode Exit fullscreen mode

Step 6: Create a function to preprocess an image before using it in a machine learning model.

Add this code in script tag below the previous function:

const preprocessImage = (imageElement: HTMLImageElement) => {
        const imageTensor = tf.browser.fromPixels(imageElement);
        const resizedImageTensor = tf.image.resizeBilinear(
            imageTensor,
            [224, 224]
        );
        return resizedImageTensor;
    };
Enter fullscreen mode Exit fullscreen mode

Explanation of the preprocessImage function
Feature: Preprocess an image before using it in a machine learning model.

Inputs:

imageElement: An HTMLImageElement representing an image in the HTML DOM.

Steps:

Convert the image to 3D tensor (height, width, color channels) with tf.browser.fromPixels.
Resize the image to 224x224 pixels using bilinear interpolation: tf.image.resizeBilinear.

Output:

A tensor representing the preprocessed image, ready for use in a machine learning model.

Step 7: Create a function to make a prediction

Add this code in script tag below the previous function:

const classifyImage = async (
        model: mobilenet.MobileNet,
        preprocessedImage: tf.Tensor3D
    ) => {
        const predictions = await model.classify(preprocessedImage);
        return predictions;
    };
Enter fullscreen mode Exit fullscreen mode

Step 8: Put it all together

Now we need to combine the functions we created to load the model and make predictions. Add this code in script tag below the previous function:

let predictionsResult: Prediction[] = [];

const predictionImage = async (imgDisplay: HTMLImageElement) => {
        const model = await loadModel();
        const preprocessedImage = preprocessImage(imgDisplay);
        predictionsResult = await classifyImage(model, preprocessedImage);
    };
Enter fullscreen mode Exit fullscreen mode

Step 9: Create an user interface with Flowbite Svelte and Tailwind CSS.

I put here the complete code of the +page.svelte file. I added some elements like a spinner or a TypeScript interface.

<script lang="ts">
    import * as tf from '@tensorflow/tfjs';
    import * as mobilenet from '@tensorflow-models/mobilenet';
    import {
        Label,
        Fileupload,
        Heading,
        List,
        Li,
        Card,
        Spinner
    } from 'flowbite-svelte';

    let fileuploadprops = {
        id: 'inputImage'
    };

    let predictionsResult: Prediction[] = [];
    let imgDisplay: string;
    let files: FileList;
    let imgtoprocess: HTMLImageElement;
    let model: mobilenet.MobileNet;
    let predictionsLoading = false;

    $: if (files) {
        imgDisplay = URL.createObjectURL(files[0]);
    }

    interface Prediction {
        className: string;
        probability: number;
    }

    const loadModel = async () => {
        const version = 2;
        const alpha = 0.5;
        model = await mobilenet.load({ version, alpha });
        console.log('Model loaded successfully');
        return model;
    };

    const preprocessImage = (imageElement: HTMLImageElement) => {
        const imageTensor = tf.browser.fromPixels(imageElement);
        const resizedImageTensor = tf.image.resizeBilinear(
            imageTensor,
            [224, 224]
        );
        return resizedImageTensor;
    };

    const classifyImage = async (
        model: mobilenet.MobileNet,
        preprocessedImage: tf.Tensor3D
    ) => {
        const predictions = await model.classify(preprocessedImage);
        return predictions;
    };

    const predictionImage = async (imgDisplay: HTMLImageElement) => {
        predictionsLoading = true;
        const model = await loadModel();
        const preprocessedImage = preprocessImage(imgDisplay);
        predictionsResult = await classifyImage(model, preprocessedImage);
        predictionsLoading = false;
    };

    const handleImageChange = () => {
        predictionImage(imgtoprocess);
        URL.revokeObjectURL(imgDisplay);
    };
</script>

<svelte:head>
    <title>ML Vision</title>
    <meta
        name="description"
        content="A simple ML Vision app"
    />
</svelte:head>

<section class="flex flex-col gap-12">
    <Heading tag="h1" class="text-center">ML Vision with MobileNet images Classifier</Heading>
    <div class="ml-auto mr-auto block w-1/2">
        <Label class="pb-2 text-lg">Select an image to classify</Label>
        <Fileupload
            {...fileuploadprops}
            bind:files
            on:change={handleImageChange}
        />
    </div>
    <Card
        size="md"
        padding="sm"
        class="ml-auto mr-auto"
    >
        <img
            hidden={!imgDisplay}
            src={imgDisplay}
            alt=""
            bind:this={imgtoprocess}
        />
        <div class="py-4">
            <Heading tag="h3">Predictions:</Heading>
            <List
                position="outside"
                class="p-4"
            >
                {#if predictionsLoading}
                    <span class="flex flex-col items-center"
                        ><Spinner color="blue" /></span
                    >
                {:else}
                    {#each predictionsResult as prediction}
                        <Li
                            >{prediction.className}: {Math.round(
                                prediction.probability * 100
                            )}%</Li
                        >
                    {/each}
                {/if}
            </List>
        </div>
    </Card>
</section>
Enter fullscreen mode Exit fullscreen mode

Step 10: Add Svelte components

I create a Footer and Menu components and display them in +layout.svelte file.

  • Create a file ./src/lib/components/footer/Footer.svelte and add this code: 
<script>
    import {
        Footer,
        FooterCopyright,
    } from 'flowbite-svelte';
</script>

<Footer class="rounded-none bg-black p-4 shadow md:px-6 md:py-8">
    <hr class="my-6 border-white sm:mx-auto lg:my-8" />
    <FooterCopyright
        aClass="text-[#6C70ED] text-base"
        spanClass="text-white block text-sm text-center"
        href="/"
        by="Flowbite"
    />
</Footer>
Enter fullscreen mode Exit fullscreen mode
  • Create a file ./src/lib/components/menu/Menu.svelte and add this code: 
<script lang="ts">
    import {
        Navbar,
        NavBrand,
        NavLi,
        NavUl,
        NavHamburger
    } from 'flowbite-svelte';
</script>

<Navbar
    let:hidden
    let:toggle
    class="bg-black text-white"
>
    <NavBrand href="/">
    </NavBrand>
    <NavHamburger on:click={toggle} class="hover:bg-[#6C70ED]" menuClass="outline-[#6C70ED]" />
    <NavUl {hidden} ulClass="bg-black flex flex-col p-4 mt-4 md:flex-row md:space-x-8 rtl:space-x-reverse md:mt-0 md:text-sm md:font-medium" divClass="w-full md:block md:w-auto">
        <NavLi
            href="/"
            active={true} class="md:text-white md:hover:text-[#6C70ED]" nonActiveClass="text-white">Home</NavLi
        >
    </NavUl>
</Navbar>
Enter fullscreen mode Exit fullscreen mode
  • Then edit the ./src/routes/+layout.svelte file with this code: 
<script>
    import Menu from '$lib/components/menu/Menu.svelte';
    import Footer from '$lib/components/footer/Footer.svelte';
    import '../app.css';
</script>

<div class="flex flex-col min-h-screen">
    <Menu />

    <main class="flex flex-col flex-1 p-4 w-full box-border max-w-5xl mt-0 mb-0 ml-auto mr-auto">
        <slot />
    </main>

    <Footer />
</div>
Enter fullscreen mode Exit fullscreen mode

Step 11: Try your project

To run your project:

npm run dev -- --open
Enter fullscreen mode Exit fullscreen mode

Choose an image with a dog or a vegetable. See the result!

Conclusion

In this article, you learned how to create a simple image recognition website with SvelteKit and TensorFlow.js. This project is a great starting point for learning about machine learning in the browser.

Project on Github

https://github.com/Estelle-K/ml-vision

Top comments (4)

Collapse
 
aditya_raj_1010 profile image
A.R •

Considering the implementation of the image recognition website using SvelteKit and TensorFlow.js, discuss potential optimizations or enhancements that could be applied to improve the user experience or the overall performance of the machine learning model inference process, especially when dealing with larger datasets or more complex models. Additionally, explore how you would handle potential challenges related to scalability and responsiveness in a real-world scenario.

followback for more insightful discussion

Collapse
 
estelle-k profile image
Estelle-K •

Thank you for your recommendations. I hope develop this project throughout my journey in machine learning.

Collapse
 
aditya_raj_1010 profile image
A.R •

all the best

Collapse
 
codingmaster97 profile image
codingmaster97 •

Wow, I'm impressed

Some comments may only be visible to logged-in visitors. Sign in to view all comments.

Read next

a_gaoba profile image

Understanding Array Prototypes in JavaScript

Abdulrahman Gaoba -

itsrorymurphy profile image

Version Control: Git Basics

Rory Murphy -

jguerrero-voxel51 profile image

Annotation is dead

Jimmy Guerrero -

surjendu104 profile image

A Beginner's Guide to Flask App Development: Getting Started with Python's Microframework

SURJENDU PAL -