Wicked audio feedback loops in the browser with PureScript

In this post, I'm going to show you how to use PureScript to create audio feedback loops. Feedback is great for creating dynamic and surprising soundscapes in interactive audio environments and video games. You can hear a demo here. If you're on a desktop browser, make sure to use headphones! Tested on desktop Firefox and mobile Chrome 🚀

We'll be using purescript-wags and purescript-halogen to create our project. Let's get started!

tl;dr git clone https://github.com/mikesol/feedback && cd feedback && npm install && npm start

Hello wags

The Wags in purescript-wags stands for "WebAudio Graphs As a Stream." You can think of the Web Audio API like a classic mixing board that pipes IO from inputs (ie microphones) to busses (ie a reverb or compressor) to the main fader. Imagine if you filmed a mixing board at 60 frames per second, gave it to an engineer and said "use this information to reproduce a mix". That's what Wags does: it samples changes in a mixing board and uses those changes to construct a sequence of low-level commands to the web audio API.

Our "mixing board" is represented by a Row Type in PureScript. Each track or bus has a name (index) and a value. In our case, the value will be an audio unit, like a high-pass filter, and potentially other audio units that are bussed into this one.

Here's a representation of audio tracks in ProTools.

And here are tracks using Wags. Like ProTools, the tracks have labels, and like ProTools, they are organized vertically. For example, the master gain unit called mix connects to the speaker, three aux busses (gain0, gain1, gain2) connect to the mix, etc.

type SceneType
  = { speaker :: TSpeaker /\ { mix :: Unit }
    , mix :: TGain /\ { gain0 :: Unit, gain1 :: Unit, gain2 :: Unit }
    -- feedback0
    , gain0 :: TGain /\ { microphone :: Unit, hpf0 :: Unit, delay_1_2 :: Unit }
    , hpf0 :: THighpass /\ { delay0 :: Unit }
    , delay0 :: TDelay /\ { atten0 :: Unit }
    , atten0 :: TGain /\ { gain0 :: Unit }
    -- feedback1
    , gain1 :: TGain /\ { microphone :: Unit, bpf1 :: Unit }
    , bpf1 :: TBandpass /\ { delay1 :: Unit }
    , delay1 :: TDelay /\ { atten1 :: Unit }
    , atten1 :: TGain /\ { gain1 :: Unit, delayX :: Unit }
    -- feedback2
    , gain2 :: TGain /\ { microphone :: Unit, hpf2 :: Unit }
    , hpf2 :: THighpass /\ { delay2 :: Unit }
    , delay2 :: TDelay /\ { atten2 :: Unit }
    , atten2 :: TGain /\ { gain2 :: Unit }
    -- intermediary feedback
    , delay_1_2 :: TDelay /\ { gain_1_2 :: Unit }
    , gain_1_2 :: TGain /\ { gain2 :: Unit, gain1 :: Unit }
    -- full loop
    , delayX :: TDelay /\ { mix :: Unit }
    -- microphone
    , microphone :: TMicrophone /\ {}
    }

The type above is a blueprint for our mixer. Next, let's see how to change audio parameters over time to create some gnarly effects.

Sound effects

The code below moves the faders on our mixing board. We first get time from the environment and then use it to modulate certain busses in our mixer. Specifically, we'll modulate the delay busses, which creates something that sounds like pitch shifting, and we'll also modulate the filters, which creates a sweeping effect.

type FrameTp p i o a
  = Frame (SceneI Unit Unit) FFIAudio (Effect Unit) p i o a

doChanges :: forall proof. FrameTp proof SceneType SceneType Unit
doChanges = WAGS.do
  { time } <- env
  ivoid
    $ change
        { hpf0: highpass_ { freq: ap' $ sin (time * pi * 0.5) * 1000.0 + 1500.0 }
        , delay0: delay_ $ ap' (0.4 + sin (time * pi * 2.0) * 0.2)
        , bpf1: bandpass_ { freq: ap' $ cos (time * pi * 1.6) * 1000.0 + 1500.0 }
        , delay1: delay_ $ ap' (0.3 + cos (time * pi * 0.7) * 0.1)
        , hpf2: highpass_ { freq: ap' $ cos (time * pi * 4.0) * 1000.0 + 1500.0 }
        , delay2: delay_ $ ap' (2.0 + sin (time * pi * 0.2) * 1.6)
        }

Putting it all together

To wrap things up, let's build our mixing board. createFrame will start with an empty mixing board {} and build up the board described by SceneType using a command patch from the Wags API. Then change (also from Wags) initializes certain static values and doChanges initializes the rest of them. Lastly, in piece, we loop doChanges ad infinitum.

createFrame :: FrameTp Frame0 {} SceneType Unit
createFrame =
  patch
    :*> change
        { atten0: gain_ 0.6
        , gain0: gain_ 0.5
        , atten1: gain_ 0.6
        , gain1: gain_ 0.5
        , atten2: gain_ 0.6
        , gain2: gain_ 0.5
        , gain_1_2: gain_ 0.7
        , delay_1_2: delay_ 2.0
        , mix: gain_ 1.0
        }
    :*> doChanges

piece :: Scene (SceneI Unit Unit) FFIAudio (Effect Unit) Frame0
piece =
  createFrame
    @|> loop (const doChanges)

And that's it! Our mixer is ready to mix 🎧 🔊

Conclusion

Using purescript-wags, we are able to design mixing boards at the type level, create them using patch, change them using change, and loop those changes using loop. The entire repo can be cloned & ran locally by executing:

git clone https://github.com/mikesol/feedback
cd feedback
npm install
npm start

In addition to wags, the repo uses halogen for the presentation layer, tailwind for CSS and webpack + postcss to bundle everything up.