Introduction
In this article, I'll show you how to build a simple website that allows users to upload an image and get predictions about its content using MobileNet model from TensorFlow.js. This project is perfect for beginners in TensorFlow.js who want to get started with machine learning in the browser.
Prerequisites
Before you start, you should have some knowledge of web development and be familiar with JavaScript, TypeScript and Tailwind CSS. Make sure the following tools are also installed:
- npm or yarn
- VSCode (https://code.visualstudio.com/download)
- Svelte extension (https://marketplace.visualstudio.com/items?itemName=svelte.svelte-vscode)
Step 1: Create a new SvelteKit project
Start by creating a new SvelteKit project using the following command in the terminal:
npm init svelte@next my-project
Step 2: Install Tailwind CSS and Flowbite Svelte
- Tailwind CSS is a CSS framework. https://tailwindcss.com/
- Flowbite Svelte is an official Flowbite component library for Svelte. https://flowbite-svelte.com/
Install Tailwind CSS in a SvelteKit project in the terminal:
npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p
Update your tailwind.config.js
file:
/** @type {import('tailwindcss').Config} */
export default {
content: ['./src/**/*.{html,js,svelte,ts}'],
theme: {
extend: {}
},
plugins: []
};
Create ./src/app.css
file:
@tailwind base;
@tailwind components;
@tailwind utilities;
Create a ./src/routes/+layout.svelte
fle and importe the app.css
file:
<script>
import "../app.css";
</script>
<slot />
Install Flowbite Svelte in the terminal:
npm i -D flowbite-svelte flowbite
Update your tailwind.config.js
file:
/** @type {import('tailwindcss').Config} */
export default {
content: [
'./src/**/*.{html,js,svelte,ts}',
'./node_modules/flowbite-svelte/**/*.{html,js,svelte,ts}'
],
theme: {
extend: {
colors: {
// flowbite-svelte
primary: {
50: '#FFF5F2',
100: '#FFF1EE',
200: '#FFE4DE',
300: '#FFD5CC',
400: '#FFBCAD',
500: '#FE795D',
600: '#EF562F',
700: '#EB4F27',
800: '#CC4522',
900: '#A5371B'
}
}
}
},
plugins: [require('flowbite/plugin')]
};
Step 3: Install TensorFlow.js
- TensorFlow.js documentation https://www.tensorflow.org/js
- Pre-trained TensorFlow.js models https://github.com/tensorflow/tfjs-models#readme
Install TensorFlow.js in the terminal:
npm i @tensorflow-models/mobilenet @tensorflow/tfjs @tensorflow/tfjs-vis
Step 4: Import the MobileNet model
Go to ./src/routes/+page.svelte
and delete everything there.
Create a <script lang="ts"></script>
tag and add these imports:
<script lang="ts">
import * as tf from '@tensorflow/tfjs';
import * as mobilenet from '@tensorflow-models/mobilenet';
</script>
Step 5: Create a function to load the model
Create a function to load the MobileNet model and return it. Add this code in script
tag under imports:
let model: mobilenet.MobileNet;
const loadModel = async () => {
const version = 2;
const alpha = 0.5;
model = await mobilenet.load({ version, alpha });
console.log('Model loaded successfully');
return model;
};
Step 6: Create a function to preprocess an image before using it in a machine learning model.
Add this code in script
tag below the previous function:
const preprocessImage = (imageElement: HTMLImageElement) => {
const imageTensor = tf.browser.fromPixels(imageElement);
const resizedImageTensor = tf.image.resizeBilinear(
imageTensor,
[224, 224]
);
return resizedImageTensor;
};
Explanation of the preprocessImage function
Feature: Preprocess an image before using it in a machine learning model.
Inputs:
imageElement
: An HTMLImageElement
representing an image in the HTML DOM.
Steps:
Convert the image to 3D tensor (height, width, color channels) with tf.browser.fromPixels
.
Resize the image to 224x224 pixels using bilinear interpolation: tf.image.resizeBilinear
.
Output:
A tensor representing the preprocessed image, ready for use in a machine learning model.
Step 7: Create a function to make a prediction
Add this code in script
tag below the previous function:
const classifyImage = async (
model: mobilenet.MobileNet,
preprocessedImage: tf.Tensor3D
) => {
const predictions = await model.classify(preprocessedImage);
return predictions;
};
Step 8: Put it all together
Now we need to combine the functions we created to load the model and make predictions. Add this code in script
tag below the previous function:
let predictionsResult: Prediction[] = [];
const predictionImage = async (imgDisplay: HTMLImageElement) => {
const model = await loadModel();
const preprocessedImage = preprocessImage(imgDisplay);
predictionsResult = await classifyImage(model, preprocessedImage);
};
Step 9: Create an user interface with Flowbite Svelte and Tailwind CSS.
I put here the complete code of the +page.svelte
file. I added some elements like a spinner or a TypeScript interface.
<script lang="ts">
import * as tf from '@tensorflow/tfjs';
import * as mobilenet from '@tensorflow-models/mobilenet';
import {
Label,
Fileupload,
Heading,
List,
Li,
Card,
Spinner
} from 'flowbite-svelte';
let fileuploadprops = {
id: 'inputImage'
};
let predictionsResult: Prediction[] = [];
let imgDisplay: string;
let files: FileList;
let imgtoprocess: HTMLImageElement;
let model: mobilenet.MobileNet;
let predictionsLoading = false;
$: if (files) {
imgDisplay = URL.createObjectURL(files[0]);
}
interface Prediction {
className: string;
probability: number;
}
const loadModel = async () => {
const version = 2;
const alpha = 0.5;
model = await mobilenet.load({ version, alpha });
console.log('Model loaded successfully');
return model;
};
const preprocessImage = (imageElement: HTMLImageElement) => {
const imageTensor = tf.browser.fromPixels(imageElement);
const resizedImageTensor = tf.image.resizeBilinear(
imageTensor,
[224, 224]
);
return resizedImageTensor;
};
const classifyImage = async (
model: mobilenet.MobileNet,
preprocessedImage: tf.Tensor3D
) => {
const predictions = await model.classify(preprocessedImage);
return predictions;
};
const predictionImage = async (imgDisplay: HTMLImageElement) => {
predictionsLoading = true;
const model = await loadModel();
const preprocessedImage = preprocessImage(imgDisplay);
predictionsResult = await classifyImage(model, preprocessedImage);
predictionsLoading = false;
};
const handleImageChange = () => {
predictionImage(imgtoprocess);
URL.revokeObjectURL(imgDisplay);
};
</script>
<svelte:head>
<title>ML Vision</title>
<meta
name="description"
content="A simple ML Vision app"
/>
</svelte:head>
<section class="flex flex-col gap-12">
<Heading tag="h1" class="text-center">ML Vision with MobileNet images Classifier</Heading>
<div class="ml-auto mr-auto block w-1/2">
<Label class="pb-2 text-lg">Select an image to classify</Label>
<Fileupload
{...fileuploadprops}
bind:files
on:change={handleImageChange}
/>
</div>
<Card
size="md"
padding="sm"
class="ml-auto mr-auto"
>
<img
hidden={!imgDisplay}
src={imgDisplay}
alt=""
bind:this={imgtoprocess}
/>
<div class="py-4">
<Heading tag="h3">Predictions:</Heading>
<List
position="outside"
class="p-4"
>
{#if predictionsLoading}
<span class="flex flex-col items-center"
><Spinner color="blue" /></span
>
{:else}
{#each predictionsResult as prediction}
<Li
>{prediction.className}: {Math.round(
prediction.probability * 100
)}%</Li
>
{/each}
{/if}
</List>
</div>
</Card>
</section>
Step 10: Add Svelte components
I create a Footer and Menu components and display them in +layout.svelte
file.
- Create a file
./src/lib/components/footer/Footer.svelte
and add this code:
<script>
import {
Footer,
FooterCopyright,
} from 'flowbite-svelte';
</script>
<Footer class="rounded-none bg-black p-4 shadow md:px-6 md:py-8">
<hr class="my-6 border-white sm:mx-auto lg:my-8" />
<FooterCopyright
aClass="text-[#6C70ED] text-base"
spanClass="text-white block text-sm text-center"
href="/"
by="Flowbite"
/>
</Footer>
- Create a file
./src/lib/components/menu/Menu.svelte
and add this code:
<script lang="ts">
import {
Navbar,
NavBrand,
NavLi,
NavUl,
NavHamburger
} from 'flowbite-svelte';
</script>
<Navbar
let:hidden
let:toggle
class="bg-black text-white"
>
<NavBrand href="/">
</NavBrand>
<NavHamburger on:click={toggle} class="hover:bg-[#6C70ED]" menuClass="outline-[#6C70ED]" />
<NavUl {hidden} ulClass="bg-black flex flex-col p-4 mt-4 md:flex-row md:space-x-8 rtl:space-x-reverse md:mt-0 md:text-sm md:font-medium" divClass="w-full md:block md:w-auto">
<NavLi
href="/"
active={true} class="md:text-white md:hover:text-[#6C70ED]" nonActiveClass="text-white">Home</NavLi
>
</NavUl>
</Navbar>
- Then edit the
./src/routes/+layout.svelte
file with this code:
<script>
import Menu from '$lib/components/menu/Menu.svelte';
import Footer from '$lib/components/footer/Footer.svelte';
import '../app.css';
</script>
<div class="flex flex-col min-h-screen">
<Menu />
<main class="flex flex-col flex-1 p-4 w-full box-border max-w-5xl mt-0 mb-0 ml-auto mr-auto">
<slot />
</main>
<Footer />
</div>
Step 11: Try your project
To run your project:
npm run dev -- --open
Choose an image with a dog or a vegetable. See the result!
Conclusion
In this article, you learned how to create a simple image recognition website with SvelteKit and TensorFlow.js. This project is a great starting point for learning about machine learning in the browser.
Top comments (4)
Considering the implementation of the image recognition website using SvelteKit and TensorFlow.js, discuss potential optimizations or enhancements that could be applied to improve the user experience or the overall performance of the machine learning model inference process, especially when dealing with larger datasets or more complex models. Additionally, explore how you would handle potential challenges related to scalability and responsiveness in a real-world scenario.
followback for more insightful discussion
Thank you for your recommendations. I hope develop this project throughout my journey in machine learning.
all the best
Wow, I'm impressed
Some comments may only be visible to logged-in visitors. Sign in to view all comments.