DEV Community

Cover image for Journal your ideas and experiences with your voice: Voice Journal
Prashik Meshram
Prashik Meshram

Posted on

Journal your ideas and experiences with your voice: Voice Journal

This is a submission for the Cloudflare AI Challenge.

What I Built

Voice Journal is a journal app which transcribes your natural speech to what I call a voice note 🔉. Apart from this, it also stores your voice recordings for playback ⏯️.

It has built-in AI chat integration, so you can ask anything about your notes, like list the activities I did today. You can also generate a concise summary of your voice note.


Here, try the live version of Voice Journal deployed on Cloudflare Pages and AI Workers.

VoiceJournal Demo

My Code

Here’s the GitHub repository

The project is structured into two directories: web for frontend and worker for AI workers.


It was my first time using Cloudflare AI Workers, and I had an amazing developer experience working with it. I started building the app with AI Workers and used Hono as a routing library.

Apart from this, I was getting an error while working with the whisper model via the REST API; thus, later, I tried the same but with Cloudflare AI Bindings for Workers and it worked.

Thus, I had to deploy the workers separately for the same issue.

Build the frontend with Next.js, Tailwind CSS and shadcn components, integrated the AI Worker, and finally deploy it on Cloudflare Pages.

I have integrated three AI models, here’s the detail for each:

The whisper model is used for generating transcripts from audio; it takes an array buffer converted to an 8-bit unsigned integer for audio processing.

For text-generation, integrated mistral-7b-instruct-v0.1 model.

To provide a summary of voice notes, the bart-large-cnn model is used, which tries to summarize the recorded voice notes. I think it needs more fine-tuning for better results.

The most challenging part of this whole building process was handling the audio data format and how to process it, as it was my first time dealing with it.

The future plan is to connect Cloudflare R2 object storage to store the audio files and fine-tune the summarization model.

Multiple Models and/or Triple Task Types

The Voice Journal utilizes three models for performing the tasks:

  1. whisper for speech to text conversion
  2. mistral-7b-instruct-v0.1 for text generation
  3. bart-large-cnn to summarize the notes

Top comments (4)

shikkaba profile image

The AI isn't really necessary honestly. It's like AI for the sake of AI. I asked it how many notes I made, and it gave me a rambly answer and an estimate of 30 to 40 notes. There were 5. That was kinda funny.

prashik profile image
Prashik Meshram • Edited

Currently, it acts as an AI chatbot, because I have not provided the AI model with context of user's notes. Still working on that...

Edit: It's fixed now.

fretny profile image

This sounds interesting, waiting for fine tune

prashik profile image
Prashik Meshram

Thanks, I also think that the summarization model is in beta, that's why the results are all alright.