Mobile Web Audio: Removing Media Controls from Notifications Tray

Introduction

In the world of web development, even the tiniest details can significantly impact user experience. Recently, while working on improving React ChatBotify, I encountered a persistent issue concerning its notifications feature. This problem had lingered since the initial release, but was relegated to the sidelines as other features took precedence.

So what's the issue? While notification sounds played fine for Desktop users, a subtle inconvenience emerged for mobile users: notification sounds triggered the emergence of media controls within the device's notifications tray. Take a look below to see what I mean:

Feature-breaking? No. Annoying? Yes! In the latest version of this library however, I finally resolved this annoyance once and for all.

The Original Implementation

Before delving into my attempts to rectify the issue, I thought it'd help to share some context on the notifications feature. Basically, the chatbot permits toggling of notification sounds for incoming messages. The library achieves this by initially setting up the notification audio and subsequently triggering it when necessary. For those curious about the inner workings, here's the function responsible for setting up notifications:

const setUpNotifications = () => {
  setNotificationToggledOn(botOptions.notification?.defaultToggledOn as boolean);

  let notificationSound = botOptions.notification?.sound;

  if (notificationSound?.startsWith("data:audio")) {
    const binaryString = atob(notificationSound.split(",")[1]);
    const arrayBuffer = new ArrayBuffer(binaryString.length);
    const uint8Array = new Uint8Array(arrayBuffer);
    for (let i = 0; i < binaryString.length; i++) {
      uint8Array[i] = binaryString.charCodeAt(i);
    }
    const blob = new Blob([uint8Array], { type: "audio/wav" });
    notificationSound = URL.createObjectURL(blob);
  }

  notificationAudio.current = new Audio(notificationSound);
  notificationAudio.current.volume = botOptions.notification?.volume as number;
}

Additionally, here's the function responsible for playing the notifications:

const handleNotifications = () => {
  // if embedded, or no message found, no need for notifications
  if (botOptions.theme?.embedded || messages.length == 0) {
    return;
  }
  const message = messages[messages.length - 1]
  // yes i know these conditions are ugly but i will clean this up someday ;-;
  if (message != null && message?.sender !== "user" && !isBotTyping
    && (!botOptions.isOpen || document.visibilityState !== "visible")) {
    setUnreadCount(prev => prev + 1);
    if (!botOptions.notification?.disabled && notificationToggledOn && hasInteracted) {
      notificationAudio.current?.play();
    }
  }
}

In essence, I loaded the audio and adjusted its volume upon initialization, triggering its playback when handleNotifications was invoked. Though functional, the appearance of media controls in mobile devices' notification trays was a thorn in the flesh!

Attempt #1

Hoping for a quick remedy, I did a swift Google search that led me to the controls attribute, which appeared to be what I needed. Surely, setting controls to false would remove the media controls from the notifications tray right? With a hint of optimism, I threw in a single line at the end of setUpNotifications:

notificationAudio.current.controls = false

Much to my dismay, it didn't work. The media controls were still appearing in the notifications tray.

Attempt #2

Since I was using the HTML 5 Audio, I decided to look into its documentation provided. That was when I stumbled upon controlsList, which also seemed like a promising attribute. Seeking to replace controls with controlsList, I was stumped when I noticed that there was no built-in controlsList attribute for my notificationAudio. With a "let's just try" attitude, I added controlsList directly as an attribute instead:

notificationAudio.current.setAttribute("controlsList", "nodownload");

Hmmmmm, not too unexpectedly, it's not working!

Attempt #3

As I combed the internet for more inspirations, I came upon a suggestion to call .load() again on the audio after playing it. I then had the idea to reset the audio after playing it to hopefully make the media controls go away. Feeling a little confident of this idea, I went into handleNotifications and added a line right after I called .play():

notificationAudio.current?.load()

Anddddd nope, still not working!!

Attempt #4

Feeling a little desperate and restless, I decided to try adding an <audio> tag directly and then using controlsList within it. Hopping over to the return block responsible for the render, I plonked in the following line:

<audio ref={notificationAudio} controlsList="nodownload" />

Ughhhhh! As you may have guessed, it's still not working.

Attempt #5

Exhausted and having tried out various approaches without much success, I decided to give Web Audio API a try. While it appeared slightly harder to work with, I was frankly running out of options. All that said, moving from HTML5 Audio to Web Audio API involved quite a fair bit of changes and experimentation.

Without boring you with the details of my struggles, my final changes involved me ditching notificationAudio and replacing it with audioContextRef, audioBufferRef and gainNodeRef as you can see below:

// audio to play for notifications
 const audioContextRef = useRef<AudioContext | null>(null);
 const audioBufferRef = useRef<AudioBuffer>();
 const gainNodeRef = useRef<AudioNode | null>(null);

Needless to say, I also made significant updates to both functions for setting up and playing notification sounds:

/**
 * Sets up the notifications feature (initial toggle status and sound).
 */
const setUpNotifications = async () => {
  setNotificationToggledOn(botOptions.notification?.defaultToggledOn as boolean);

  let notificationSound = botOptions.notification?.sound;
  audioContextRef.current = new AudioContext();
  const gainNode = audioContextRef.current.createGain();
  gainNode.gain.value = botOptions.notification?.volume || 0.2;
  gainNodeRef.current = gainNode;

  let audioSource;
  if (notificationSound?.startsWith("data:audio")) {
    const binaryString = atob(notificationSound.split(",")[1]);
    const arrayBuffer = new ArrayBuffer(binaryString.length);
    const uint8Array = new Uint8Array(arrayBuffer);
    for (let i = 0; i < binaryString.length; i++) {
      uint8Array[i] = binaryString.charCodeAt(i);
    }
    audioSource = arrayBuffer;
  } else {
    const response = await fetch(notificationSound as string);
    audioSource = await response.arrayBuffer();
  }

  audioBufferRef.current = await audioContextRef.current.decodeAudioData(audioSource);
}

/**
 * Handles notification count update and notification sound.
 */
const handleNotifications = () => {
  // if no audio context or no messages, return
  if (!audioContextRef.current || messages.length === 0) {
    return;
  }

  const message = messages[messages.length - 1]
  // if message is null or sent by user or is bot typing, return
  if (message == null || message.sender === "user" || isBotTyping) {
    return;
  }

  // if chat is open but user is not scrolling, return
  if (botOptions.isOpen && !isScrolling) {
    return;
  }

  setUnreadCount(prev => prev + 1);
  if (!botOptions.notification?.disabled && notificationToggledOn && hasInteracted && audioBufferRef.current) {
    const source = audioContextRef.current.createBufferSource();
    source.buffer = audioBufferRef.current;
    source.connect(gainNodeRef.current as AudioNode).connect(audioContextRef.current.destination);
    source.start();
  }
}

Given the substantial amount of changes, I had to tinker around and try out various attempts first. At some point, the notifications even stopped sounding off entirely, which felt like an even larger setback. However, with a bit more testing and determination, the notification sound eventually chimed again!

Needless to say, pulling down the notifications tray, and not seeing the media controls anymore was extremely satisfying! I am very satisfied with the outcome and while I cannot be certain that this is the best approach, this at least worked out for me. I was also a little surprised that I could not find a solution easily online. Perhaps, I'm just not looking in the right place!

Conclusion

I hope this article along with the code snippets I shared above will be helpful to anyone out there grappling with similar issues. While I've shared my journey in tackling the pesky problem of media controls showing up on mobile devices' notification trays, I'm still open to ideas, suggestions and feedback - especially if you know a better solution!

Feel free to leave your thoughts in the comments below or reach out anytime. If you've read so far, thank you for tuning in to my struggles. Catch you later, and happy coding!