How does AI bounding box detection work?

🧠 Learn in 30 seconds

It looks so simple when AI does it right?

But the AI doesn't give you an image, it gives you data. It's up to you to make it look simple. The results you get are presented in a way so you can choose specific items from all the returned results.

You might think a box gives you four values, and you're right, but it only gives you TWO points. From that you can infer a box to draw that identifies the object.

However... it very rarely gives you one box. Lots of times you get a whole bunch of boxes for you to chose from.

This is because the model is often trained to return X results no matter what, and sometimes it's meant to detect a whole wide array of objects, not just one.

Many boxes are identified by what they are detecting, like people or tables etc. But you might have a photo with three people in it, and it returns 30 people boxes. Each of these boxes generally give you a score of confidence, and it's your job to identify if they are identifying the same object, or separate objects.

There's some well known algorithms to take the best of the best.

Once you have the boxes you're interested in, you can draw them easily with a canvas. In JavaScript you just overlay a canvas on the image and draw rectangles there.

The final result lets you draw clean object detection on websites. It's pretty cool and easy to do this on images and video.

Want the code to do this? It's all in Chapter 6 of my book with a deep explanation.

I'm looking forward to what you can do with your very own AI object detection! Don't forget to tag your cool stuff with the #MadeWithTFJS Hashtag on Twitter!

And don't forget to follow me for more AI tidbits like this!