Overview of My Submission
With the submitted app Health Assistant users can enter vital parameters with their voice and store it easily electronically.
The program can gather these body parameters
- Blood Pressure
- Glucose
- Heart Rate
- Temperature
- Weight
To put in the data a kind of chat is used to ask the user for missing info and to provide feedback to the user.
After you start the app, start the recognition and just say
My weight was 86 kg today at 8 O'clock.
If the intent and the necessary parameters where recognized the app asks for a confirmation and stores the record. If not all necessary data is given directly the program will ask for it.
It is also possible to control the app. For example to display an overview of the stored data just say
Show weight
The image shows an example for such a conversation to record data.
Submission Category: Accessibility Advocates
I used this category because the app should help people with no experience with computers or disabilities to store important health data in an electronic record so that they can share it easily with physicians.
Link to Code on GitHub
The code for this sample app is shared in this Git repository. Here you can also find more technical details.
Content
This repository contains a MAUI app that was developed in a Hackathon which was started to develop solutions which benefit from Deepgram's speech recognition service The user can put in health data like heart rate, temperature or weight in a kind of chat. The utterances of the user are translated into text from the Deepgram service, and the program extracts the relevant info from this text and stores it. In case some info is missing the app asks the user to put in that data.
General Info
The app is a prototype and was developed in a short timeframe. Therefore you should not expect ready to roll-out code with outstanding code quality.
Nevertheless the app works and can be used to test the analysis approach and the behavior of the Deepgram API If you want to see it in action check out this video on YouTube
Technical details
General
โฆ
License
The code is licensed under the Apache-2 permissive license. Please find the license file in the repository.
Additional Resources / Info
The app in the repository can only be built with the preview version of Visual Studio 2022.
Check out the video in my YouTube channel to see the app in action.
Dev-Deepgram-Hackathon Demo
Improvements
For sure, there is always room for improvement and especially with such a tight schedule and a lot of new technology to learn.
The results would be even better if a specialized model in Deepgram is used.
I'm not sure whether the approach to process/analyze the text that the Deepgram service has recognized is sufficient. Usually, people would create some kind of AI for that but my educated guess is, that this is not necessary - ELIZA worked also without any AI. With this approach the latency for analysis and recognition by the Deepgram service deliver a fluent interaction for the user. More sophisticated analyses would take more time and may disturb the flow.
Clearly not all possible input is taken into account in the analysis, and it has to be analyzed what utterances are used. Unfortunately (at least from this point of view) the recognized text is not available in the Deepgram dashboard of the service.
Top comments (1)
This is really cool! I love the use case.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.