This is a submission for Twilio Challenge v24.06.12
Welcome to Chappy, your versatile chat companion! You can access Chappy through your web browser or directly via WhatsApp, enjoying seamless synchronization from WhatsApp to WebApp.
Chappy is a true multitasker. Not only can it read images, text, and voice chats, but it also excels in providing detailed and accurate responses to your enquiries. Whether you snap a photo of a document, send a voice message, or type out a question, Chappy can interpret and understand the content, delivering helpful information in return.
Moreover, Chappy is designed to hold meaningful conversations. It remembers the context of your previous interactions, which allows it to offer more personalized and relevant responses. This context-awareness ensures that Chappy can follow complex discussions and provide coherent answers, making your experience smoother and more intuitive.
The WhatsApp Bot is crafted with:
- Twilio WhatsApp API
- Next.js API
- Transcribe API for deciphering voice notes
- Vercel for hosting
- MongoDB for storing all your chats
- Google Gemini 1.5 Flash Model for advanced capabilities
The Web Application shines with:
- Next.js for a smooth user experience
- Twilio Verify API for secure login
- Twilio WhatsApp API to send chats from the web app to WhatsApp
- Vercel for reliable hosting
- MongoDB to keep all your conversations safe
- Google Gemini 1.5 Flash Model for enhanced functionality
- Speech recognition and text-to-speech conversion, powered by Polly
Demo
Please be patient with the chatbot as it may take some time to respond. I’m utilizing freely available AI models and haven’t subscribed to any paid services.
WhatsApp Twilio Sandbox
Web App: https://chappy-lyart.vercel.app
Github Repo: https://github.com/ketanrajpal/chappy
Screenshots
Twilio and AI
We leveraged Twilio's capabilities in conjunction with advanced AI technologies to create an efficient and seamless chat experience with Chappy. Here's how:
Twilio WhatsApp API: The Twilio WhatsApp API is at the core of Chappy's communication framework. It enables smooth and reliable messaging services between users and Chappy on WhatsApp. By integrating this API, Chappy can send and receive text, images, and voice chats, ensuring that users can interact with Chappy in a versatile and intuitive manner.
Transcribe API: The Transcribe API is crucial for deciphering voice notes. When a user sends a voice message, the API transcribes the audio into text, which Chappy can then process and respond to accurately. This feature enhances accessibility and convenience, allowing users to communicate with Chappy using voice.
Google Gemini 1.5 Flash Model: The integration of the Google Gemini 1.5 Flash Model equips Chappy with advanced AI capabilities. This model enhances Chappy's ability to understand and process natural language, making responses more accurate and contextually relevant. It also powers features like speech recognition and text-to-speech conversion, enriching the overall user experience.
Speech Recognition and Text-to-Speech Conversion: Powered by Polly, these features allow Chappy to convert spoken language into text and vice versa. Users can send voice messages that Chappy transcribes and responds to in text, or they can receive spoken responses from Chappy, making interactions more dynamic and accessible.
By combining Twilio's robust messaging APIs with cutting-edge AI technologies, we've created a powerful, versatile, and user-friendly chat companion in Chappy.
Additional Prize Categories
Twilio Times Two: Integrates Twilio WhatsApp API for messaging and Twilio Verify API for secure login, showcasing versatile use of Twilio's capabilities.
Impactful Innovators: Enhances user experience through seamless communication across WhatsApp and WebApp platforms, potentially benefiting accessibility and usability.
Entertaining Endeavors: While primarily functional, its advanced AI integration could lead to creative user interactions and applications, potentially qualifying it for innovative and engaging user experiences.
Top comments (0)