HealthifAI — 𝘚𝘦𝘢𝘮𝘭𝘦𝘴𝘴 𝘏𝘦𝘢𝘭𝘵𝘩𝘤𝘢𝘳𝘦 𝘴𝘰𝘭𝘶𝘵𝘪𝘰𝘯𝘴 𝘧𝘰𝘳 𝘗𝘳𝘰𝘷𝘪𝘥𝘦𝘳𝘴 🏥⚕️

Category Submission:

• Wacky Wildcard 🃏
• Smooth Shifters 🌬️

---

What we built 🤗

HealthifAI is a smart Web application built to provide Seamless Healthcare solutions for Providers & is fueled by Linode. 🏥⚕️

Creators :

• Pratyay Banerjee (@neilblaze)
• Subham Sahu (@subhamx)

Inspiration 💡

Healthcare is one of the most important and critical industries in the world. Providing quality medical care to patients is essential, but it is often hindered by various challenges such as overburdened healthcare workers, lack of medical devices in rural areas, and administrative stress.

With the advent of artificial intelligence and machine learning, the healthcare industry has a unique opportunity to tackle these challenges head-on and revolutionize the way medical care is delivered.

With this as context, we plan to tackle the Provider Shortage & Burnout and Access to Care strategic themes.

What it does 🤔

HealthifAI aims to tackle several key pain points in the healthcare industry — specifically for the following :

Provider Shortage & Burnout :

• Intuitive, easy & safe digital patient record entry which eliminates the need for manual and legacy record entry methods.
• We provide an ML-powered "soft diagnosis" to save time for doctors and nurses.
• We have location-based COVID-19 alerts to better equip workers.
• Multilingual speech-to-text notes, because it's easier!
• Reminder system to help with medication/check-ups. Keeping track of everything is hard!

Access to care :

• Multilingual communication model that transcribes speech from any language into English. This is particularly helpful in rural areas where communication is a barrier.
• Experimental Computer-Vision powered heart rate monitor. This transforms everyday hand-held devices into medical devices - an exciting vision for the future!

---

App Tryout Link 🔗

👉 Home : https://healthifai-with.tech [Frontend deployed on Vercel ▲ & Backend deployed on Linode]

Alternate URL : http://45.79.166.94 (Hosted on Linode)

📌 Endpoint List :

• https://healthifai-with.tech/login → Login page
• https://healthifai-with.tech/result → Bad Request (4xx)
• https://healthifai-with.tech/result?sym1=cough&sym2=itching&sym3=none&sym4=none&sym5=none → Disease Prediction [Note : Passing "none" is allowed, provied we have to pass all five symptoms. Feel free to explore list of symptoms over here. Methods allowed : POST & GET]

• https://healthifai-with.tech/transcribe → OpenAI Whisper Auto-Translate EN Transcription [ You can view the uploaded recdummy.mp3 file recorded in Hindi (HI) language here. Methods allowed : POST & GET]

• https://healthifai-with.tech/d3data?sym1=cough&sym2=itching&sym3=none&sym4=none&sym5=none → Returns only precautions.

Video ▶️

Privacy & Security 🔐

HealthifAI handles a wide range of sensitive information as healthcare data. In the wrong hands, this data could dramatically harm individuals. We took special efforts and considerations to ensure that our platform protects the privacy and sensitive information of all of our users making it 100% GDPR compliant!

How we built it ⚙️

HealthifAI was built using cutting-edge AI and Machine Learning technologies, including OpenAI's whisper as well as DETR (End-to-End Object Detection) model with ResNet-50 backbone.

For the disease detection part, we've used a Kaggle dataset which can be found here. Our machine learning model performs extremely well in terms of disease prediction as we benchmarked an accuracy of more than 92% over a prolonged period. Based on the inference, we also return symptom severity and basic precautions in no time!

We used the Flask framework to build a RESTful API that can handle incoming requests and return appropriate responses. For the front-end, we used React.JS & Tailwind as CSS framework. The Authentication (OAuth) has been done by Firebase & we’re also using the Cloudstore database for storing user logs. We have deployed the front-end of our Webapp on Vercel & most importantly, the backend is running on Linode.

The API was integrated with the Open-AI speech-to-text model "whisper" to transcribe speech from any language into English. Further, Gaussian Naive Bayes for classification was implemented to "soft diagnose" patients based on their symptoms.

We're also running our custom algorithm to analyse and return the heartbeat in realtime using a concept called as photoplethysmography, where we leverage a camera & with the capability of face detection, we record images of facial skin, as skin can represent changes in arterial blood volume between the systolic and diastolic phases of the cardiac cycle & then we return the ROI. The computer-vision powered heart rate monitor was built using image processing techniques built with OpenCV. In essence, the camera detects sensitive changes in the neck and forehead which is then used to infer heart rate.

---

Screenshots 🖼️

Link to Source Code 👨‍💻

Permissive License ⚖️

MIT

Why Linode?

Building scalable systems are always a tricky thing. Although our app is not serving millions of customers as of now, but as software enthusiast, we strive to build an infinitely scalable application. And here Linode helped us a lot.

Thanks to Linode for providing us with $100 credits! 😊

Linode is a cloud-based virtual machine service that we used to deploy the backend of our HealthifAI project. We chose Linode because it provides a reliable and scalable hosting solution for our web application, and it also offers competitive pricing.

Once our virtual machine was up and running, we installed and configured the necessary softwares and other tools required for our application. We also set up security measures, including firewalls and SSL certificates, to ensure that our backend was protected from potential cyber threats.

Linode provided us with an easy-to-use interface to manage our virtual machine, including monitoring tools to track server performance and resource usage. It also allowed us to scale our resources up or down depending on the demands of our application.

Overall, using Linode allowed us to deploy a reliable and scalable backend for our HealthifAI project, without worrying about the complexities of managing our own physical servers.

---

Design 🎨

We were heavily inspired by the revised version of Double Diamond design process, a model popularized by the British Design Council, which not only includes visual design, but a full-fledged research cycle in which you must discover and define your problem before tackling your solution & then finally deploy it.

1. Discover: a deep dive into the problem we are trying to solve.
2. Define: synthesizing the information from the discovery phase into a problem definition.
3. Develop: think up solutions to the problem.
4. Deliver: pick the best solution and build that.

Moreover, we utilized design tools like Figma, Photoshop & Illustrator to prototype our designs before doing any coding. Through this, we are able to get iterative feedback so that we spend less time re-writing code.

Research 📚

Research is the key to empathizing with users: we found our specific user group early and that paves the way for our whole project. Here are a few of the resources that were helpful to us —

• What do healthcare workers spend most time on? | NIH
• Measuring Heart-rate through muted videos | MIT Media lab
• An overview of healthcare in rural areas | Rural Health Information
• Provider Burnout | NIH
• Communication in rural healthcare | Optimizing rural healthcare
• Can we use ML to diagnose diseases? | NIH
• Lack of medical workers plagues developing world | Reuters
• Linode docs
• Linode Compute Instances

CREDITS

• Design Resources : Freepik, Behance
• Icons : Icons8, fontawesome
• Font : Righteous / Roboto / Raleway

---

Challenges we ran into 😤

Building HealthifAI was not without its challenges. One of our challenges was integrating the various AI and machine learning technologies into a cohesive and functional system. This required a deep understanding of each technology, as well as expertise in data processing and software engineering. We participated in hourly review sessions to share findings of distributed research - our biggest challenge was sticking to tight schedules! Moreover, we also ran into troubles while deploying the backend on Linode, but thanks to it's amazing documentation, things got sorted quite quickly!

We are proud of finishing the project on time which seemed like a tough task as we started working on it quite late due to other commitments. We were also able to add most of the features that we envisioned for the app during ideation. And as always, working overnight was pretty fun! :)

What's next? 🚀

The sky's the limit for HealthifAI. We are already exploring new ways to improve and expand the platform, including incorporating new technologies and partnering with healthcare providers to bring our vision to a wider audience. We're committed to making a real impact in the healthcare industry and changing lives for the better.

Conclusion 🐣

That's it for now! We can't wait to see the impact that HealthifAI will have on the world. Stay tuned for updates and more exciting developments! Also, I would love to thank my project partner @subhamx for helping me, & Special thanks goes to @devencourt for resolving everyone's doubts! 🙌.

And as always, thank you #DEV #DEVCommunity & #Linode for hosting this hackathon! 💚