Category Submission:

• Choose Your Own Adventure ⚛
• Google Cloud Superstar ☁️

What we built 🤔

Binoculearn is a bleeding-edge smart P2P educational video conferencing web application aimed to deliver a reliable frame rate & is backed by low-latency support along with low jitter (smooth and consistent), as well as high audio quality. We do this by converting the video stream into ASCII characters on the client side and send it via WebRTC using Twilio’s video conferencing service deployed on Google Cloud App Run ☁️ & is fuelled by MongoDB Atlas! 🍃

💡 Implementing video conferencing using this technique saves bandwidth bidirectionally & especially on the receiver end. This method is both vertically & horizontally scalable as we can feed more users as they enter the conference.

App Tryout Link 🔗

Binoculearn.Ai 👉 https://binoculearn-5jjb23a4oq-ue.a.run.app [Deployed on Google Cloud App Run ☁️]

Features 🎠

• P2P lagfree video conferencing app with ultra-low bandwidth support!
• Bleeding-edge Image Compression Algorithm!
• Vertically & Horizontally Scaleble [Currently capped at 4, because of Twilio Credits]
• Twilio Live Transcription [Stored in MongoDB Atlas]
• P2P Messaging with Sentiment Analysis via Natural Language API!
• Generate Summary & Transcript of the meeting!
• File Sharing (blob) via MongoDB Atlas!
• User Dashboard with Previous Activity Tracker!
• Minimalist UI/UX powered by ReactJS & Tailwind CSS
• High Quality Multiplexed Audio!
• Overall Meeting Emotion Tracker
• MongoDB Atlas as Non-SQL DB
• Deployed on Google Cloud App Run
• Saves 💰 + Internet Data!
• Secure O-Auth via Firebase by Google!
• 100% GDPR compliant & SEO friendly inteface!

Video ▶️

Privacy & Security 🔐

Binoculearn deals with a wide range of sensitive information. In the wrong hands, this data could dramatically harm individuals. We took special efforts and considerations to ensure that our platform protects the privacy and sensitive information of all of our users making it 100% GDPR compliant!

We also made sure that all data is sent securely over the network. Binoculearn leverages the security benefits of TLS for encryption. We also encoded all of our data using Base64 encoding. Ideally, in a future iteration, we would like to encrypt all data using a more secure method.

Background 📜

As ubiquitous and fast as the internet seems in developed countries, developing countries still struggle with reliable internet connections. The impact of poor internet connectivity exasperate the education inequality between children from prosperous countries and children from developing countries, because the latter cannot benefit from remote learning via video conferencing. 😔

Last year, it was found that millions of students in the state of Odisha in India are stuck at home with no access to either internet or online education. My friend (teammate) Subham Sahu, an Odisha native, has had first-hand experience of interruptions during his undergraduate studies.

For even those who have access to the internet, the price is premium and the bandwidth is limited. For instance, while talking to his parents in India, Subham found that they frequently run out of their allocated 1 GB far before the allowance period, after which the bandwidth gets throttled: stalled frames, choppy audio, painful delays, and eventual disconnections, and subsequent retries are a normal occurrence, but still arguably much better than normal telephone conversations because he gets to “see” them.

At the heart of the problem may lack tele-infrastructure for the implementation of education on virtual platforms.

Why RGB Frames are heavy?	Why of What?

Videos are array of images that consists of large bytes of data which packets transfer from one place to another. Using our smart algorithm, we are compressing the image and then converting it into grayscale bit images which are then transfused by ASCII characters while transmitting. 🪄

To address this problem, we propose a new approach based on the insight that if we are willing to give up some realism or realistic rendering of faces and screens, then there is a whole new world of face and screen representations that can be derived for ultra-low bandwidth, with an acceptable quality of experience! 👪

The proposed solution can be primarily implemented as software needing no change in the underlying infrastructure! This would in turn be cheaper, and allow internet access to people that are currently being marginalized based on their affordability.

We aim to prioritize a reliable frame rate with low latency and low jitter (smooth and consistent), as well as high audio quality serving the purpose of online education for the better! ✨

Our goal with this platform is to connect students in poorly connected areas with highly qualified teachers in metropolitan areas and abroad to facilitate Cost-Effective Stable Remote Collaboration.

Screenshots 🖼️

Home Page [Before O-Auth]	Home Page [After O-Auth]

Host a Meeting [Old / 16:9]	Join a Meeting [Old / 16:9]

Host a Meeting [New]	Join a Meeting [New]

User Dashboard [New]	Dashboard Insight [New]

Description 🦄

Experience a Superfast, low-latency P2P videochat even on ultra-low Bandwidth networks. Redefining the communication gap, Binoculearn is a MIT licensed open-sourced project made for students, by the students & will be Free Forever! ⚡

On top of the bandwidth-saving functionality, we also offer educational and content-moderation tools like Sentiment Analysis via Google Cloud's Natural Language API for session chat QnA and our custom Trained ML model deployed on GCP for Meeting Summarization.

These features allow both the educators and students to maintain decorum in the meeting and also have follow-up material to retain information about the meeting!

Link to Source Code 👨‍💻

git

Permissive License ⚖️

MIT

How we built it ⚙️

First and foremost, it is Crafted with 💙. The whole process can be broken into the following points :-

• ➤ React.JS, Redux + Tailwind CSS on the frontend
• ➤ Express.js, Node.js, Sockets, WebRTC, Twilio Live on the backend

• ➤ Prisma for connecting the Frontend to the MongoDB Atlas to store user Data + Logs
• ➤ External services like Twilio, GCP Natural Language API
• ➤ GitHub as CI/CD and Google's App Engine for Deployment [Duplicate Deployment done on Vercel]

QnA model Architecture BERT:	Summarizer Architecture

➤ Prisma Schema for MongoDB Atlas:

➤ Database Deployments for MongoDB Atlas [Click Below .Gif ⬇️]:

➤ Data Services — MongoDB Atlas [Click Below .Gif ⬇️]:

➤ Collections — MongoDB Atlas [Binoculearn]:

➤ Session Timeline-Chart MongoDB Atlas [Binoculearn]:

➤ Session Sentiment — Timeline MongoDB Atlas [Binoculearn]:

Why MongoDB Atlas?

Databases are always a tricky thing, as they're stateful in nature. Although our app is not serving millions of customers, but as software enthusiast, we strive to build an infinitely scalable application. And here MongoDB Atlas helped a lot.

We didn't have to manage the database, no need to work on complex networking between shards, etc. (And thanks to Google Cloud Run, our server can scale too. Hopefully this app gets viral and we're able to cater a million real users! 😊)

We used MongoDB as our primary database to store user sessions, meeting details, user socket IDs (which powers our real-time websockets engine), etc.

We're also generating summary of the meeting, and instead of generating in an synchronous flow, we're using MongoDB realm to have a serverless function hosted to generate the summary. In this way, we're able to keep the app more cohesive! 🙂

We learnt a lot of things including most of the MongoDB Realm, and many of the features of MongoDB Atlas. Data modelling in MongoDB was bit tricky, as both Neel and I come from transitional relational database space. We together studied few of the video lectures from MongoDB University to understand the anti patterns and in-depth data modelling in MongoDB.

Apart from MongoDB we learnt about Google App Engine, Cloud run, and Cloud build. 🌟

Design 🎨

We were heavily inspired by the revised version of Double Diamond design process, a model popularized by the British Design Council, which not only includes visual design, but a full-fledged research cycle in which you must discover and define your problem before tackling your solution & then finally deploy it.

1. Discover: a deep dive into the problem we are trying to solve.
2. Define: synthesizing the information from the discovery phase into a problem definition.
3. Develop: think up solutions to the problem.
4. Deliver: pick the best solution and build that.

Moreover, we utilized design tools like Figma, Photoshop & Illustrator to prototype our designs before doing any coding. Through this, we are able to get iterative feedback so that we spend less time re-writing code.

Research 📚

Research is the key to empathizing with users: we found our specific user group early and that paves the way for our whole project. Here are a few of the resources that were helpful to us —

• https://msutoday.msu.edu/news/2020/poor-internet-connection-leaves-rural-students-behind
• https://www.theguardian.com/technology/2021/nov/30/more-than-a-third-of-worlds-population-has-never-used-the-internet-says-un
• https://bit.ly/3UCmir3
• https://towardsdatascience.com/video-calling-for-billions-without-internet-40d10069c464
• https://www.retrium.com/blog/dont-let-slow-internet-connections-ruin-your-retrospectives
• https://www.statista.com/chart/17247/the-average-cost-of-mobile-data-in-selected-countries/
• https://www.broadbandsearch.net/blog/internet-statistics
• https://www.speechly.com/blog/create-a-webrtc-video-chat-app-with-speechly-transcription
• https://aclanthology.org/2020.lrec-1.825.pdf
• https://www.currentscience.ac.in/Volumes/110/01/0069.pdf
• Prisma Docs : https://www.prisma.io/docs/guides/database/using-prisma-with-mongodb
• Atlas Docs : https://www.mongodb.com/docs/atlas
• GCP Natural Language API with MongoDB Atlas : https://youtu.be/4DoU32EHC8c?t=3131
• ReactJS Docs : https://reactjs.org/docs/getting-started.html
• https://iimskills.com/the-future-of-online-education-in-india
• https://www.ucf.edu/online/leadership-management/news/why-the-future-of-learning-is-digital-and-for-everyone

CREDITS

• Design Resources : Freepik, Behance
• Icons : Icons8, fontawesome
• Font : Urbanist / Roboto / Raleway

Challenges we ran into 😤

This project was initially built in under 24 hours, 19th Nov — 20th Nov, 2022 & I actually got to know about the MongoDB Atlas hackathon from Subham on 20th of November, 2022 (IST time) when we were eagerly waiting for MetroHack's result. Personally, I'm a Hackathon freak. I love hackathons because it helps to generate specific ideas in a distinct domain within a small span of time & as a result, it not only increases the creativity but also enhances the curiosity while tackling corner cases while building the project/product. Since then, we kept committing towards this project. We [I, Subham & Gaurang] did face some challenges during the hackathon, many of which ironically related to working remotely. One of the major challenges was the time difference. All of us participated from different time zones, which created communication challenges.

One of the biggest challenge we faced was with deployment. Since our application has a web socket endpoint, we had very few deployment options. We initially thought that App Engine standard environment would serve the purpose, but in the end realised that it doesn't support Websockets. Finally we had to migrate our deployment stack to Google Cloud Run, which provided us the flexibility of environment and also allowed to use web sockets.

We are proud of finishing the project on time which seemed like a tough task as we started working on it quite late due to other commitments. We were also able to add most of the features that we envisioned for the app during ideation. And as always, working overnight was pretty fun! :)

What's next? 🚀

We believe that our App has great potential. We just really want this project to have a positive impact on people's lives! We would love to make it more scalable & cross-platform so that the user interaction increases to a great extent! Also, it's noted that we do have a bunch of ideas in our bucket-list which we are looking forward to transform into reality!

Conclusion 🐣

It has been all fun, & I would love to thank my buddies @subhamx & Gaurang for helping me, & Special thanks goes to @stanimiravlaeva, @mlynn & @joel__lord 🙌. And as always, thank you #DEV #DEVCommunity & #MongoDB for hosting this hackathon! 💚

Update [13/12/2022] ⚠️ — Few days ago, we ran out of credits, hence API credentials had been revoked! You can run the same on your local, hence, use your own credentials. But recently, we re-deployed it via Google Cloud Run, so hopefully you can explore our live-app at — https://binoculearn-5jjb23a4oq-ue.a.run.app

Update [15/12/2022] ⚠️ — Replaced Old Demo Video with New one!