DEV Community

BeReadable - Online Multilingual Audio Transcription and Recorder

Mostafa Said on March 20, 2022

Overview of My Submission Lets start this one with a question. Does my audio need a transcript? Short answer: yes. In the world of inter...

Read full post

BekahHW • Mar 22 '22

This looks great! I can't get the livestream option to work though. It plays, but doesn't transcribe.

Mostafa Said • Mar 22 '22

It was working last time I tested it 😅 It should work only if the sound comes from external speakers and the mic is close to it. I tried to find a way to send the livestream itself over to Deepgram and receive the transcription but figured it's not possible.

BekahHW • Mar 22 '22

ahhh. ok. I had headphones plugged in. Have you checked out our "getting started with streaming docs"? This part of the docs might help too.

Mostafa Said • Mar 22 '22

Yes I tried to follow this one but I can't use SDKs in Vuejs that's why my only option was to use WebSocket to transfer the user media to Deepgram but it can't pass the live stream itself unfortunately. Thanks for helping 🙏🙏

SandraRodgers • Mar 23 '22

Hi Mostafa!

We were chatting about this earlier today and it was suggested that it would be preferable for your livestream to fetch the stream data using the stream url, rather than record the stream with the microphone.

There's an example on the docs of fetching this BBC livestream. Look at the node.js example on the docs here:

developers.deepgram.com/documentat....

The node code for fetch is:

fetch(url).then(r => r.body).then(res => {
    res.on('readable', () => {
        if(deepgramLive.getReadyState() == 1) {
            deepgramLive.send(res.read())
        }
    })
})

However, since you are doing the transcription on the frontend, you could use the fetch API.

I have an example codepen of a basic fetch request here: codepen.io/sandrarodgers/pen/VwzrZPe

You've probably used fetch before, but just wanted to provide an example. I think you should try fetching the livestream directly and sending that to Deepgram to transcribe. That's my suggestion!

Your project is awesome by the way! It looks so great!

Mostafa Said • Mar 23 '22

Omg! This is exactly what I was looking for. I'm glad to hear that it can work from the frontend side with fetch, I will work on this as soon as I get home and maybe update the UI and release a modified version of it. Thank you so much 🙏😊

SandraRodgers • Mar 23 '22

You're welcome!

Mostafa Said • Mar 29 '22

@sandrarodgers I'm stuck :D Fetch can't send my stream data to Deepgram from front end. I've been told it's not possible to send the stream itself from the front end that's why I gave up on this option in the first place (stackoverflow.com/questions/714894... )
I found something I'm trying to use but I believe this will be a different way to do it. I will fetch the stream and play it in my browser and send audio to Deepgram from my browser after converting it ( gist.github.com/revolunet/e620e2c5... )

SandraRodgers • Mar 29 '22

You mean a stream like the hosted BBC stream, right? Or do you mean your stream that is being captured by the browser microphone?

Let me know which one, and I'll try to send you some code examples.

Mostafa Said • Mar 29 '22

Yes stream like the hosted BBC.
The one captured by browser microphone is working fine in my project as you can see in live transcription section but the live stream section I want it to send the live stream link like the one hosted by BBC to deepgram and receive transcription.

SandraRodgers • Mar 29 '22

Yes, it is trickier than I first said. I was able to do it though.

Here is an example. You need to change the API key to your key. And you might need to refresh a couple times. Open the console and make sure there isn't an error. But you should see the text transcript show up on the screen.

stackblitz.com/edit/web-platform-v...

SandraRodgers • Mar 29 '22

This example takes the BBC url, fetches the data that comes back as a ReadableStream (the response body), and converts that to data to send to Deepgram:

const url =
    'https://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk';
  fetch(url)
    .then((response) => response.body)
    .then((body) => {
      // use method to parse audio data
      readAllChunks(body);
    });

The function readAllChunks takes the response (which is a ReadableStream) and uses the getReader() to convert it into audio data that can be sent to Deepgram. So instead of using response.json() like in my basic fetch example, it uses response.getReader() to deal with that data type.

async function readAllChunks(readableStream) {
  const reader = readableStream.getReader();
  const chunks = [];
  let done, value;
  while (!done) {
    ({ value, done } = await reader.read());
    socket.send(value);
    if (done) {
      return chunks;
    }
    chunks.push(value);
  }
}

I was able to write this because I found this stack overflow question that helped me.

Tell me if this works for you! Good luck!

Mostafa Said • Mar 29 '22

Awesome! I spent hours trying to do it I'm so glad that it's not impossible. Will work on it and let you know of the updates.

Thank you so much for helping :)

Mostafa Said • Mar 29 '22

Below is the final VueJS method that I tried this morning. Will delete it and start over with your code and I hope it will work for me.

Thanks again :)

const sendStream = () => {
      console.log("starting");
      const url =
        "http://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk";

      const language = document.querySelector("select").value;
      const socket = new WebSocket(
        "wss://api.deepgram.com/v1/listen?language=" + language,
        ["token", process.env.VUE_APP_DEEPGRAM_KEY]
      );

      socket.onopen = () => {
        console.log("socket opened");

        fetch(url)
          .then((response) => response.body)
          .then((data) => {
            socket.send(data.getReader());
          });
      };

      socket.onmessage = (message) => {
        const received = JSON.parse(message.data);
        const transcript = received.channel.alternatives[0].transcript;
        console.log(transcript);
      };
    };