DEV Community

Cover image for BeReadable - Online Multilingual Audio Transcription and Recorder

BeReadable - Online Multilingual Audio Transcription and Recorder

Mostafa Said on March 20, 2022

Overview of My Submission Lets start this one with a question. Does my audio need a transcript? Short answer: yes. In the world of inter...
Collapse
 
bekahhw profile image
BekahHW

This looks great! I can't get the livestream option to work though. It plays, but doesn't transcribe.

Collapse
 
moose_said profile image
Mostafa Said

It was working last time I tested it šŸ˜… It should work only if the sound comes from external speakers and the mic is close to it. I tried to find a way to send the livestream itself over to Deepgram and receive the transcription but figured it's not possible.

Collapse
 
bekahhw profile image
BekahHW

ahhh. ok. I had headphones plugged in. Have you checked out our "getting started with streaming docs"? This part of the docs might help too.

Thread Thread
 
moose_said profile image
Mostafa Said

Yes I tried to follow this one but I can't use SDKs in Vuejs that's why my only option was to use WebSocket to transfer the user media to Deepgram but it can't pass the live stream itself unfortunately. Thanks for helping šŸ™šŸ™

Thread Thread
 
sandrarodgers profile image
SandraRodgers

Hi Mostafa!

We were chatting about this earlier today and it was suggested that it would be preferable for your livestream to fetch the stream data using the stream url, rather than record the stream with the microphone.

There's an example on the docs of fetching this BBC livestream. Look at the node.js example on the docs here:

developers.deepgram.com/documentat....

The node code for fetch is:

fetch(url).then(r => r.body).then(res => {
    res.on('readable', () => {
        if(deepgramLive.getReadyState() == 1) {
            deepgramLive.send(res.read())
        }
    })
})
Enter fullscreen mode Exit fullscreen mode

However, since you are doing the transcription on the frontend, you could use the fetch API.

I have an example codepen of a basic fetch request here: codepen.io/sandrarodgers/pen/VwzrZPe

You've probably used fetch before, but just wanted to provide an example. I think you should try fetching the livestream directly and sending that to Deepgram to transcribe. That's my suggestion!

Your project is awesome by the way! It looks so great!

Thread Thread
 
moose_said profile image
Mostafa Said

Omg! This is exactly what I was looking for. I'm glad to hear that it can work from the frontend side with fetch, I will work on this as soon as I get home and maybe update the UI and release a modified version of it. Thank you so much šŸ™šŸ˜Š

Thread Thread
 
sandrarodgers profile image
SandraRodgers

You're welcome!

Thread Thread
 
moose_said profile image
Mostafa Said

@sandrarodgers I'm stuck :D Fetch can't send my stream data to Deepgram from front end. I've been told it's not possible to send the stream itself from the front end that's why I gave up on this option in the first place (stackoverflow.com/questions/714894... )
I found something I'm trying to use but I believe this will be a different way to do it. I will fetch the stream and play it in my browser and send audio to Deepgram from my browser after converting it ( gist.github.com/revolunet/e620e2c5... )

Thread Thread
 
sandrarodgers profile image
SandraRodgers

You mean a stream like the hosted BBC stream, right? Or do you mean your stream that is being captured by the browser microphone?

Let me know which one, and I'll try to send you some code examples.

Thread Thread
 
moose_said profile image
Mostafa Said

Yes stream like the hosted BBC.
The one captured by browser microphone is working fine in my project as you can see in live transcription section but the live stream section I want it to send the live stream link like the one hosted by BBC to deepgram and receive transcription.

Thread Thread
 
sandrarodgers profile image
SandraRodgers

Yes, it is trickier than I first said. I was able to do it though.

Here is an example. You need to change the API key to your key. And you might need to refresh a couple times. Open the console and make sure there isn't an error. But you should see the text transcript show up on the screen.

stackblitz.com/edit/web-platform-v...

Thread Thread
 
sandrarodgers profile image
SandraRodgers

This example takes the BBC url, fetches the data that comes back as a ReadableStream (the response body), and converts that to data to send to Deepgram:

const url =
    'https://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk';
  fetch(url)
    .then((response) => response.body)
    .then((body) => {
      // use method to parse audio data
      readAllChunks(body);
    });
Enter fullscreen mode Exit fullscreen mode

The function readAllChunks takes the response (which is a ReadableStream) and uses the getReader() to convert it into audio data that can be sent to Deepgram. So instead of using response.json() like in my basic fetch example, it uses response.getReader() to deal with that data type.

async function readAllChunks(readableStream) {
  const reader = readableStream.getReader();
  const chunks = [];
  let done, value;
  while (!done) {
    ({ value, done } = await reader.read());
    socket.send(value);
    if (done) {
      return chunks;
    }
    chunks.push(value);
  }
}
Enter fullscreen mode Exit fullscreen mode

I was able to write this because I found this stack overflow question that helped me.

Tell me if this works for you! Good luck!

Thread Thread
 
moose_said profile image
Mostafa Said

Awesome! I spent hours trying to do it I'm so glad that it's not impossible. Will work on it and let you know of the updates.

Thank you so much for helping :)

Thread Thread
 
moose_said profile image
Mostafa Said

Below is the final VueJS method that I tried this morning. Will delete it and start over with your code and I hope it will work for me.

Thanks again :)

const sendStream = () => {
      console.log("starting");
      const url =
        "http://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk";

      const language = document.querySelector("select").value;
      const socket = new WebSocket(
        "wss://api.deepgram.com/v1/listen?language=" + language,
        ["token", process.env.VUE_APP_DEEPGRAM_KEY]
      );

      socket.onopen = () => {
        console.log("socket opened");

        fetch(url)
          .then((response) => response.body)
          .then((data) => {
            socket.send(data.getReader());
          });
      };

      socket.onmessage = (message) => {
        const received = JSON.parse(message.data);
        const transcript = received.channel.alternatives[0].transcript;
        console.log(transcript);
      };
    };
Enter fullscreen mode Exit fullscreen mode
Thread Thread
 
sandrarodgers profile image
SandraRodgers

Yep, looks like you just need to turn that response.body into a data format that can be passed on to Deepgram.

Thread Thread
 
moose_said profile image
Mostafa Said

It works just fine :) bereadable.netlify.app/livestream

Thank you so much Sandra this is really awesome and huge improvement to the functionality of the tool.

Thread Thread
 
sandrarodgers profile image
SandraRodgers

Awesome! I'm so glad it works.

Thread Thread
 
bekahhw profile image
BekahHW

Your project is so beautiful!

Thread Thread
 
moose_said profile image
Mostafa Said

Thank you! It's an honor to me that you like it šŸ˜Š

Collapse
 
biomathcode profile image
Pratik sharma

Nice work Mostafa !!

Collapse
 
moose_said profile image
Mostafa Said

Thank you Pratik! :)

Collapse
 
svgatorapp profile image
SVGator

Interesting concept. Best of luck!

Collapse
 
moose_said profile image
Mostafa Said

Thanks šŸ™

Collapse
 
gajananpp profile image
Gajanan Patil

Your all submissions are great !

Collapse
 
moose_said profile image
Mostafa Said

Thank you so much :) I'm so glad you like them.