This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.

What I Built

I built a Patient Management and Recommendation System that streamlines medical data collection and provides tailored health recommendations based on patients' symptoms. The application allows users to upload audio recordings of patient details and symptoms, which are transcribed into structured text using AssemblyAI’s advanced Speech-to-Text API. The system then parses this information, categorizes the symptoms, and displays personalized recommendations for treatment using CloudflareAI.

Key features:

• 1. Audio-to-text transcription using AssemblyAI.
• 2. Symptom analysis and tailored treatment recommendations using CloudflareAI Worker.
• 3. Clean and intuitive user interface for managing patient data.
• 4. Hosted in a Dockerized environment with a PostgreSQL database for persistent storage.

Demo

The code app is under this link:

Patient list including audio upload:

Patient details:

Journey

This project was designed to simplify managing patient records and generating personalized health recommendations. The goal was to streamline the workload for healthcare professionals by enabling them to record audio notes about patients and extract key information automatically. For example, a doctor seeing multiple patients in a day can simply upload an audio file, and the application will process it to extract critical data like the patient's name, age, symptoms, and recommended treatments—saving time and ensuring accuracy in record-keeping.

How AssemblyAI Was Used

Audio-to-Text Transcription:

1. Audio-to-Text Transcription:
   Using AssemblyAI’s API, uploaded audio files containing patient details (e.g., "Patient Jane Smith, age 29, reporting fatigue and fever") are transcribed into text. The API seamlessly converts diverse medical terms and patient notes with remarkable accuracy.

2. Data Parsing:
   After transcription, key details such as the patient’s name, age, symptoms, and date are extracted using regular expressions and integrated into the database.

3. Recommendations Engine:
   Based on the parsed symptoms, the app provides structured recommendations (e.g., "Rest and hydration," "Over-the-counter medications") using CloudflareAI. These are presented in a clear and accessible format to help users take action.

AssemblyAI Challenge Prompts:

Sophisticated Speech-to-Text Application: Built an end-to-end system that goes beyond transcription to include actionable insights and a user-friendly interface.

Intelligent AI and Audio Processing

AssemblyAI Integration:

• Converts audio files into text using the AssemblyAI transcription service.
• Extracts meaningful details such as:

1. Patient name
2. Age
3. Symptoms
4. Consultation date

Cloudflare AI Integration:

• Processes symptoms via Cloudflare AI to generate recommendations.

Tools and Technologies

• AssemblyAI for transcription.
• CloudflareAI for recommendations.
• React for the front-end.
• Node.js and Express for the back-end API.
• PostgreSQL (via Docker) for database management.
• CSS for styling the user interface.

Thanks for the opportunity to showcase this project! 😊