DEV Community

Timothy Kelvin
Timothy Kelvin

Posted on

YouTube changed its playlist JSON. My reader found 0 videos, then exactly 100 📺

#webscraping #javascript #node #youtube

Two silent failures from one layout change, and a short fix for each.

Hi Everyone!

This week I built a tool that turns YouTube videos into transcripts. You can give it single videos, but the useful part is giving it a whole channel or playlist. So before any transcript work, it has to answer a simple question: which videos are in this list?

My first version answered "none". The second answered "exactly 100", for a channel with hundreds of uploads. Neither threw an error. Here's what changed on YouTube's side, and how I fixed both.

cover-playlist-0-then-100

🗺️ How a playlist reads without a browser

YouTube's own apps talk to an internal JSON API called innertube. A playlist page is one POST to youtubei/v1/browse, with the playlist ID prefixed by VL:

const res = await fetch('https://www.youtube.com/youtubei/v1/browse?prettyPrint=false', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
        context: { client: { clientName: 'WEB', clientVersion: '2.20251110.01.00', hl: 'en', gl: 'US' } },
        browseId: `VL${playlistId}`,
    }),
});
Enter fullscreen mode Exit fullscreen mode

The answer holds the first 100 entries and a continuation token. You send the token back to browse to get the next 100, and so on.

Channels work the same way, because every channel has hidden upload playlists. Take the channel ID (UC...), swap the UC for one of these prefixes, and you get a playlist you can page through:

Prefix What's in it
UU every public upload
UULF regular videos only
UUSH Shorts only
UULV past live streams only

So UCBJycsmduvYEL83R_U4JriQ (MKBHD) becomes UULFBJycsmduvYEL83R_U4JriQ for his videos without the Shorts. One small function covers channels and playlists alike.

A note before you build on this: YouTube's robots.txt disallows /youtubei/ for bots, and its terms limit automated access. Read both and decide for yourself. Plenty of tools use this API, and that doesn't make it permission.

🕳️ Failure 1: zero videos

Older code reads playlist entries from playlistVideoRenderer objects, and so did my first version. My walker searched the whole answer for that key and collected videoId from each one.

It found nothing. No error, no empty-playlist message, just an empty list that looked exactly like a real empty playlist.

So I counted every key ending in Renderer or ViewModel in the answer. playlistVideoRenderer wasn't there at all. Instead there were 100 lockupViewModel objects. Trimmed, one looks like this:

{
  "lockupViewModel": {
    "contentId": "jGwO_UgTS7I",
    "contentType": "LOCKUP_CONTENT_TYPE_VIDEO",
    "contentImage": { "thumbnailViewModel": { "...": "..." } },
    "metadata": {
      "lockupMetadataViewModel": { "title": { "content": "Lecture 1 ..." } }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The video ID moved from videoId to contentId, and the title from a runs array to a plain content string. Lockups can also hold playlists and other things, so contentType matters.

The fix reads both layouts, so it keeps working whichever one YouTube serves:

export function playlistEntries(json) {
    const out = [];
    // The old layout.
    for (const r of collect(json, 'playlistVideoRenderer')) {
        if (r?.videoId && r.isPlayable !== false) out.push({ videoId: r.videoId, title: textOf(r.title) });
    }
    // The 2026 layout.
    for (const l of collect(json, 'lockupViewModel')) {
        if (!/VIDEO/.test(l?.contentType ?? '') || !/^[A-Za-z0-9_-]{11}$/.test(l.contentId ?? '')) continue;
        out.push({ videoId: l.contentId, title: l.metadata?.lockupMetadataViewModel?.title?.content ?? null });
    }
    return out;
}
Enter fullscreen mode Exit fullscreen mode

collect(obj, key) is a small helper that returns every value stored under key anywhere in the tree. That detail matters for failure 2.

🧱 Failure 2: exactly 100

With entries fixed, a channel's uploads came back as 100 videos every time. A round number like that is never a coincidence: it's one page, and the second page never loaded.

My continuation code looked for continuationItemRenderer and read continuationEndpoint.continuationCommand.token. The new answer had a continuationItemViewModel instead, with the token one level deeper:

{
  "continuationItemViewModel": {
    "trigger": "CONTINUATION_TRIGGER_ON_ITEM_SHOWN",
    "continuationCommand": {
      "innertubeCommand": {
        "clickTrackingParams": "...",
        "continuationCommand": { "token": "4qmFsgJtEhpWTFVV..." }
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

My first fix added the new key name and searched it with collect(item, 'continuationCommand'). It still stopped at 100.

The reason was my own helper. collect stops descending once it finds the key, which is usually what you want: it doesn't return a node and then all of that node's children. Here it returned the outer continuationCommand, which has no token, and never looked inside it at the inner one, which does.

So the token search walks all the way down and takes the first continuationCommand that actually has a token:

function findToken(obj) {
    if (Array.isArray(obj)) {
        for (const v of obj) {
            const t = findToken(v);
            if (t) return t;
        }
    } else if (obj && typeof obj === 'object') {
        if (typeof obj.continuationCommand?.token === 'string') return obj.continuationCommand.token;
        for (const v of Object.values(obj)) {
            const t = findToken(v);
            if (t) return t;
        }
    }
    return null;
}

export function continuationToken(json) {
    for (const item of [...collect(json, 'continuationItemRenderer'), ...collect(json, 'continuationItemViewModel')]) {
        const token = findToken(item);
        if (token) return token;
    }
    return null;
}
Enter fullscreen mode Exit fullscreen mode

After that, asking for 250 of MKBHD's uploads returned 250, in 3 calls. Search results still use continuationItemRenderer, and the same function handles both.

🎬 Shorts are a third shape

The Shorts list (UUSH...) came back empty too, after both fixes. It uses yet another object, shortsLockupViewModel, with the video ID inside the tap action:

for (const s of collect(json, 'shortsLockupViewModel')) {
    const videoId = s?.onTap?.innertubeCommand?.reelWatchEndpoint?.videoId;
    if (videoId) out.push({ videoId, title: s.overlayMetadata?.primaryText?.content ?? null });
}
Enter fullscreen mode Exit fullscreen mode

🧪 Pin all three shapes in one test

YouTube changes these layouts without notice, and different pages use different ones: search results still came in the old shapes while playlists used the new ones. So I keep one small fixture with every shape, including the nested token, and test against it:

test('playlist answers: old and 2026 layouts, Shorts, nested continuation token', () => {
    const json = { contents: [
        { playlistVideoRenderer: { videoId: 'aaaaaaaaaaa', title: { runs: [{ text: 'Old layout' }] } } },
        { lockupViewModel: { contentId: 'ccccccccccc', contentType: 'LOCKUP_CONTENT_TYPE_VIDEO',
            metadata: { lockupMetadataViewModel: { title: { content: 'New layout' } } } } },
        { lockupViewModel: { contentId: 'PLxxxxxxxxxxxxxxxx', contentType: 'LOCKUP_CONTENT_TYPE_PLAYLIST' } },
        { shortsLockupViewModel: { onTap: { innertubeCommand: { reelWatchEndpoint: { videoId: 'ddddddddddd' } } },
            overlayMetadata: { primaryText: { content: 'A short' } } } },
        { continuationItemViewModel: { continuationCommand: { innertubeCommand: { continuationCommand: { token: 'NEXT' } } } } },
    ] };
    assert.deepEqual(playlistEntries(json).map((v) => v.videoId), ['aaaaaaaaaaa', 'ccccccccccc', 'ddddddddddd']);
    assert.equal(continuationToken(json), 'NEXT');
});
Enter fullscreen mode Exit fullscreen mode

The playlist lockup in there is on purpose: it must not come back as a video.

🚧 Other things that stopped working

While getting the transcripts themselves, I hit five more dead ends. Here they are with the exact answers, in case you're searching for one of them:

  • The web page's own caption link returned HTTP 200 with an empty body. As far as I can tell, web clients now need a proof-of-origin token for captions. YouTube's mobile app clients still got working caption links.
  • YouTube's transcript endpoint (youtubei/v1/get_transcript) answered 400 Precondition check failed.
  • Machine-translated captions (adding &tlang=es to a caption link) answered 429 from every kind of IP I tried: my home connection, datacenter and residential.
  • The TV embedded player client answered "YouTube is no longer supported in this application or device."
  • Datacenter IPs got "Sign in to confirm you're not a bot" on 10 of 12 requests. Residential IPs went through.

Takeaways

  • An empty result can be a parsing failure. When a list that should have items comes back empty, count the keys in the answer before you trust it.
  • A round number is a pagination bug. 100 results from something bigger means page two never loaded.
  • Know where your tree walker stops. A helper that stops at the first match will skip a key nested inside a key with the same name.
  • Read both layouts. Search still uses the old shapes while playlists use the new ones, so supporting both costs a few lines and saves a broken run.
  • Keep a fixture of every shape you've seen, and add to it each time YouTube surprises you.

Top comments (0)