AWS Lambda Function to insert pre-roll ads in HLS stream

In this post we will go through the principles and show some example code on how an AWS Lambda (stateless) function can be used to insert pre- and mid-roll ads in an HLS on-demand stream.

We will skip the fundamentals of HTTP based streaming why it is good to have a basic understanding of how the HLS streaming format works before reading this article.

The main principle here is that we will take a video on-demand HLS stream that we stitch together with another video on-demand HLS stream (VOD). This can be accomplished by merging the media playlists of these two on-demand streams into a new media playlist with media segments from both streams.

One media playlist from first VOD:

#EXTM3U
#EXT-X-TARGETDURATION:3
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:3.000,
ad1_0_av.ts
#EXTINF:3.000,
ad2_0_av.ts
#EXTINF:3.000,
ad3_0_av.ts
#EXTINF:3.000,
ad4_0_av.ts
#EXTINF:3.000,
ad5_0_av.ts
#EXT-X-ENDLIST

stitched together with one media playlist from the second VOD:

#EXTM3U
#EXT-X-TARGETDURATION:9
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:9.000,
segment1_0_av.ts
#EXTINF:9.000,
segment2_0_av.ts
#EXTINF:9.000,
segment3_0_av.ts
#EXTINF:9.000,
segment4_0_av.ts
#EXTINF:9.000,
segment5_0_av.ts
#EXTINF:9.000,
segment6_0_av.ts
#EXTINF:9.000,
segment7_0_av.ts
#EXTINF:9.000,
segment8_0_av.ts
#EXTINF:9.000,
segment9_0_av.ts
#EXTINF:6.266,
segment10_0_av.ts
#EXT-X-ENDLIST

would result in:

#EXTM3U
#EXT-X-TARGETDURATION:9
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:9.0000,
segment1_0_av.ts
#EXT-X-DISCONTINUITY
#EXT-X-CUE-OUT:DURATION=15
#EXTINF:3.0000,
http://mock.com/ad2/ad1_0_av.ts
#EXTINF:3.0000,
http://mock.com/ad2/ad2_0_av.ts
#EXTINF:3.0000,
http://mock.com/ad2/ad3_0_av.ts
#EXTINF:3.0000,
http://mock.com/ad2/ad4_0_av.ts
#EXTINF:3.0000,
http://mock.com/ad2/ad5_0_av.ts
#EXT-X-DISCONTINUITY
#EXT-X-CUE-IN
#EXTINF:9.0000,
segment2_0_av.ts
#EXTINF:9.0000,
segment3_0_av.ts
#EXTINF:9.0000,
segment4_0_av.ts
#EXTINF:9.0000,
segment5_0_av.ts
#EXTINF:9.0000,
segment6_0_av.ts
#EXTINF:9.0000,
segment7_0_av.ts
#EXTINF:9.0000,
segment8_0_av.ts
#EXTINF:9.0000,
segment9_0_av.ts
#EXTINF:6.2660,
segment10_0_av.ts
#EXT-X-ENDLIST

Note that we have added the tag #EXT-X-DISCONTINUITY between the two VODs. This tag instructs the player that the subsequent media segment does not have continuous timestamps. We also insert the #EXT-X-CUE-* tags to indicate that it is an ad break.

One of the challenges with the HLS standard is that it requires that the video player does at least two subsequent requests in order to be able to play a stream. The video player first downloads the master playlist/manifest that contains links to each individual media playlist. This can be troublesome when building a fully stateless API endpoint as the API does not have any memory between the individual requests.

To overcome this we will encode the state into a base64 encoded string that we pass on as a query parameter on each request. This state contains links to the VODs and how they should be stitched together. The state is represented by a JSON object that could look like this:

{
  "uri": "https://example.com/vod/master.m3u8",
  "breaks": [
    { "pos": 0, "uri": "https://example.com/ad/master.m3u8" }
  ]
}

Position 0 means that the ad to insert should be placed before the VOD.

And when we base64 encode this JSON we get:

ewogICJ1cmkiOiAiaHR0cHM6Ly9leGFtcGxlLmNvbS92b2QvbWFzdGVyLm0zdTgiLAogICJicmVha3MiOiBbCiAgICB7ICJwb3MiOiAwLCAidXJpIjogImh0dHBzOi8vZXhhbXBsZS5jb20vYWQvbWFzdGVyLm0zdTgiIH0KICBdCn0=

This base64 encoded string is then passed on with all requests as a URL query parameter.

The AWS Lambda function

The Lambda function will serve two API endpoints. One for serving the master manifest and the other for the media playlists referenced in the master manifest.

For simplicity we call these two endpoints:

• /master.m3u8
• /media.m3u8

With the example above the video player is then passed on with the following URL to play: /master.m3u8?payload=ewogICJ1cmkiOiAiaHR0cHM6Ly9leGFtcGxlLmNvbS92b2QvbWFzdGVyLm0zdTgiLAogICJicmVha3MiOiBbCiAgICB7ICJwb3MiOiAwLCAidXJpIjogImh0dHBzOi8vZXhhbXBsZS5jb20vYWQvbWFzdGVyLm0zdTgiIH0KICBdCn0=.

Pseudo code for master.m3u8

The pseudo code for the master.m3u8 endpoint to handle this request:

1. Decode payload string and unwrap into a JSON payload
2. Obtain the URI to the main VOD from the JSON payload
3. Download the master manifest for the main VOD
4. Rewrite all links in the master manifest to point to the media.m3u8 endpoint instead (including the base64 encoded payload)
5. Return this rewritten master manifest to the video player

In point no 4 we also need to extract the streams and provide an additional parameter to the media.m3u8 endpoint to determine which stream to use. For simplicity we use the bandwidth as the identifier. The URL to the media playlist will then look something like this:

/media.m3u8?bw=2500000&payload=ewogICJ1cmkiOiAiaHR0cHM6Ly9leGFtcGxlLmNvbS92b2QvbWFzdGVyLm0zdTgiLAogICJicmVha3MiOiBbCiAgICB7ICJwb3MiOiAwLCAidXJpIjogImh0dHBzOi8vZXhhbXBsZS5jb20vYWQvbWFzdGVyLm0zdTgiIH0KICBdCn0=

Pseudo code for media.m3u8

The handler to serve the media playlist is the one that will do the actual "stitching" of the VODs specified in the payload. The pseudo code for this handler will then be:

1. Decode payload string and unwrap into a JSON payload
2. Parse payload and stitch together the VODs based on the breaks array in the payload
3. Return the "stitched" media playlist to the video player

To facilitate the implementation of the logic in point no 2 we have an open sourced Javascript library @eyevinn/hls-splice that can be used. The code below shows an example of how an ad (HLS VOD) is inserted 35 seconds from the start in an HLS VOD.

const hlsVod = new HLSSpliceVod('https://maitv-vod.lab.eyevinn.technology/stswe17-ozer.mp4/master.m3u8');
hlsVod.load()
.then(() => {
  return hlsVod.insertAdAt(35000, 'https://maitv-vod.lab.eyevinn.technology/ads/apotea-15s.mp4/master.m3u8');
})
.then(() => {
  const mediaManifest = hlsVod.getMediaManifest(4928000);
  console.log(mediaManifest);
})

The AWS Lambda could then look like this:

exports.handler = async event => {
  let response;

  if (event.path === "/stitch/master.m3u8") {
    response = await handleMasterManifestRequest(event);
  } else if (event.path === "/stitch/media.m3u8") {
    response = await handleMediaManifestRequest(event);
  } else {
    response = generateErrorResponse({ code: 404 });
  }

  return response;
};

and

const handleMasterManifestRequest = async (event) => {
  try {
    const encodedPayload = event.queryStringParameters.payload;
    console.log(`Received request /master.m3u8 (payload=${encodedPayload})`);
    const manifest = await getMasterManifest(encodedPayload);
    const rewrittenManifest = await rewriteMasterManifest(manifest, encodedPayload);
    return generateManifestResponse(rewrittenManifest);
  } catch (exc) {
    console.error(exc);
    return generateErrorResponse(500);
  }
};

const handleMediaManifestRequest = async (event) => {
  try {
    const bw = event.queryStringParameters.bw;
    const encodedPayload = event.queryStringParameters.payload;
    console.log(`Received request /media.m3u8 (bw=${bw}, payload=${encodedPayload})`);
    const hlsVod = await createVodFromPayload(encodedPayload, { baseUrlFromSource: true });
    const mediaManifest = (await hlsVod).getMediaManifest(bw);
    return generateManifestResponse(mediaManifest);
  } catch (exc) {
    console.error(exc);
    return generateErrorResponse(500);
  }
};

We have now described the basic principles on how to build an AWS Lambda HTTP endpoint to insert for example pre-roll ads in an HLS on-demand stream.

If you need assistance in the development and implementation of this our team of video developers are happy to help out. If you have any questions or comments just drop a line in the comments section to this post.