Smart meal tracking on whatsapp [POC]

This is a submission for Twilio Challenge v24.06.12

What I Built

I developed a proof-of-concept WhatsApp bot that accepts an image of the meal you're having. In return, it provides the calorie intake along with detailed nutrition information such as carbohydrates, fats, and proteins.

This is just a proof of concept; we can significantly enhance it by incorporating a database to store user data and generate regular reports. Use background workers for image saving and calorie estimation from AI, and employing another worker to send the messages.

Read more about functionality and flow on github readme

Demo

I cannot provide a demo because I used ngrok to route WhatsApp webhook requests to localhost. But here are some screenshots ->

Positive case I send an image of Indian thali

Positive case I send an image of burger meal

Negative case Naruto, When he is kept away from Ramen (IYKYK)

Twilio and AI

• Twilio's API is used to receive images of food sent by users via WhatsApp.
• Google's Generative AI model - Gemeni analyzes the received images to estimate the caloric content and nutritional breakdown.
• Twilio sends back the detailed nutritional information to the user on WhatsApp.
• The integration of Twilio and AI enhances user interaction by providing instant and informative responses.

Additional Prize Categories

Twilio Times Two
Impactful Innovators

Thanks for reading my post, connect me on Linkedin @ Sanket Munot