From Audio to Diagram

This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.

What I Built

An Application that allow from an Audio concerning a discussion, a meeting, etc ... to generate a "meaningful mind-map diagram", that represent the touched key points. This representation joined with summary provide a more complete and understandable informations

Demo

The application in available here for access to full functionality you need both an AssemblyAI Api Key and a OpenAI Api Key. Below there are some representative screenshots

Settings

Upload Audio

Transcribe Audio

Generate Mindmap Diagram

Journey

To implement process from audio to diagram I have create several skilled agents described below:

1. transcribe-from-audio: this agent use AssemblyAI transcripts API to transcribe the provided audio.

2. keypoints-from-transcript: this Agent use OpenAI (got-4o-mini) to extract the Keypoints inside the given transcription

3. summary-to-mindmap: this agent use OpenAI (got-4o-mini) to arrange the key points in a kind of ontology providing a hierarchical representation of information

4. mindmap-to-mermaid: last agent transform the mind-map representation in a mermaid syntax ready for the visualization