This is a submission for Twilio Challenge v24.06.12
What Was Built
Project X is an innovative learning platform designed to revolutionize the educational experience by leveraging advanced technologies like AI and cloud computing. Users are empowered to effortlessly gather relevant information from various sources, creating an interactive and educational module. The AI Podcasts feature, powered by Project X, allows for the specification of a topic and character personas to generate engaging podcasts. Twilio APIs are used to enable interactive discussions, enhancing the learning experience.
Demo
Link to the app: https://projectx.nishithp.dev
Project X
The power of knowledge in the palm of your hands...
Execution Steps
-
Clone Frontend Repository:
- Clone the frontend repository from GitHub using the following command
git clone https://github.com/NishithP2004/projectx-frontend.git
- Clone the frontend repository from GitHub using the following command
-
Clone Backend Repository:
- Similarly, clone the backend repository from GitHub:
git clone https://github.com/NishithP2004/projectx-backend.git
- Similarly, clone the backend repository from GitHub:
-
Navigate to Frontend Directory:
- Move into the
projectx-frontenddirectory:cd projectx-frontend
- Move into the
-
Create Environment Variables File:
- Create a
.envfile in the root of theprojectx-frontenddirectory. - Copy the contents of the
.env.examplefile provided below and paste it into the.envfile. - Fill in the required values for your environment variables.
- Create a
-
Start Frontend Service:
- Run Docker Compose to start the frontend service:
docker-compose up
- Run Docker Compose to start the frontend service:
-
Navigate to Backend Directory:
- Switch to the
projectx-backenddirectory:cd ../projectx-backend
- Switch to the
-
Start Backend Service:
- Ensure you have Microsoft Azure CLI tools installed.
- Start the backend service using the Azure Functions Core Tools:
func start
.env.example (Common to both frontend & backend)
G_SEARCH_KEY=
G_CX=projectx-backend
Execution Steps
-
Clone Backend Repository:
- Clone the backend repository from GitHub using the following command
git clone https://github.com/NishithP2004/projectx-backend.git
- Clone the backend repository from GitHub using the following command
-
Navigate to Backend Directory:
- Move into the
projectx-backenddirectory:cd projectx-backend
- Move into the
-
Create Environment Variables File:
- Create a
.envfile in the root of theprojectx-backenddirectory. - Copy the contents of the
.env.examplefile provided below and paste it into the.envfile. - Fill in the required values for your environment variables. Refer to the comments in the
.env.examplefile for guidance on each variable.
- Create a
-
Install Dependencies:
- Install the necessary dependencies by running:
npm install
- Install the necessary dependencies by running:
-
Start Backend Service:
- Ensure you have Microsoft Azure CLI tools installed.
- Start the backend service using the Azure Functions Core Tools:
func start
-
Access Backend Endpoints:
- Once the backend service is running, you can access the various API endpoints provided by the backend component.
.env.example (Common to both frontend & backend)
G_SEARCH_KEY=
G_CX=
BING_CONFIG_ID=
BING_API_KEY=
GOOGLE_PALM_API_KEY=
GOOGLE_VERTEX_AI_TOKEN=Features Offered by Project X (An Overview)
Twilio and AI
In Project X, Twilio's Programmable Voice and Messages API (WhatsApp Sandbox) were utilized to implement the Interactive AI Discussions feature. This allows for the scheduling of interactive discussions, changing the direction of the discussion, and receiving a summary and transcript via WhatsApp. The AI element is powered by Azure's OpenAI (GPT-4) for generating content and Azure Speech Services for text-to-speech conversion, making the podcasts natural and engaging.
How It Works:
- Podcast Initiation: Users specify a topic and character personas for the podcast.
- Podcast Generation: Using Azure OpenAI (GPT-4), the script for the podcast is generated, grounded in the data provided by the user as part of the courses created on the platform.
- Interactive Discussion: Interaction with the podcast is enabled through Twilio APIs (barge), altering the discussion direction and resulting in a dynamic learning experience.
- Summary and Transcript: At the end of the discussion, a summary and transcript of the session are sent via WhatsApp. The same will be made available in the online platform as a normal podcast for future use.
Additional Prize Categories
Twilio Times Two
Two Twilio APIs are utilized in our submission:
- Twilio Programmable Voice: For interactive podcast discussions.
- Twilio Messages API (WhatsApp Sandbox): For delivering summaries and transcripts to users.
Impactful Innovators
A significant educational gap is addressed by Project X by providing dynamic, interactive learning resources that go beyond traditional static references. This positive impact on education can enhance the learning process, making it more engaging and effective for students.
Entertaining Endeavors
A creative and entertaining way to learn is introduced by the AI Podcasts feature. By allowing the specification of character personas, including fictional or real-world entities, the platform makes learning fun and engaging. The interactive discussions add an element of unpredictability and excitement to the learning process.
Imagine quantum physics is the topic of interest, but instead of a dry lecture, a podcast featuring Albert Einstein and Sheldon Cooper from "The Big Bang Theory" is chosen. As the podcast starts, Einstein provides deep insights into quantum theory, while Sheldon frequently interrupts with quirky, humorous comments, asking, "Why does everything have to be so... theoretical? Can't we just solve it with string cheese and a whiteboard?"
Halfway through the podcast, the direction of the discussion is changed using Twilio’s interactive feature. The conversation is steered toward the practical applications of quantum physics in everyday life. Sheldon, being Sheldon, starts brainstorming how quantum entanglement could be used to improve Wi-Fi speeds in his apartment. The podcast concludes with Einstein trying to keep a straight face while explaining the serious implications of Sheldon's idea.
At the end of this hilarious and informative session, a summary and transcript are received via WhatsApp, complete with Einstein’s theoretical explanations and Sheldon’s comical commentary. This not only makes learning quantum physics enjoyable but also leaves a memorable and engaging learning experience.
Conclusion
The AI Podcasts feature of Project X, powered by Twilio and Azure AI technologies, transforms the traditional learning experience into an interactive, engaging, and impactful journey. By leveraging Twilio's capabilities, a platform that not only educates but also entertains has been created, pushing the boundaries of creativity in the educational landscape.
Disclaimer: This project utilizes a free trial account along with the Twilio WhatsApp Sandbox API & Twilio Voice API, which requires a verified phone number for service access.
Top comments (0)