Redirects, and their Effect on Performance or How a (Seemingly Minor) Third Party Change Affected…

Redirects, and their Effect on Performance or How a (Seemingly Minor) Third Party Change Affected the Website Performance of a Popular JavaScript Bundler

When a server redirects a request — the browser has to make a second request for the file, adding (at least) one additional round trip, and delaying the delivery of the content to the end user.

As I was sitting around one day, I wondered ‘what is the maximum number of redirects on a single page?’ HTTP Archive to the rescue. In the summary_pages table, there is a numRedirects field. And, running a quick search in BigQuery, I discovered a page with over 1100 redirects:

In WebPageTest, a yellow line means that a redirect was requested. I count (ok..a keyword search for ‘302’ found) 1,187 redirects. On one page… All pointing to Gravatar’s blank avatar. #perfmatters pic.twitter.com/SCqzG6rQEG

— Doug Sillars 🇧🇪 (@dougsillars ) October 23, 2018

Of course, it is doing some weird denial of service attack on Gravatar’s default image — all of the 1100 redirects (ok, all of the ones I looked at — I got bored pretty quickly) are pointed at the Gravatar skeleton image, which is a little odd.

An Unexpected Site

As I continued down the list of websites with a large number of redirects, I saw a familiar name in the top 10. I discovered that the homepage for webpack (https://webpack.js.org) had over 600 redirects on its homepage.

Webpack is a bundling tool for JavaScript, and has many ways to help optimize the delivery of content (see Web Performance Optimization with webpack). For this reason, I was surprised to see so many redirects on this page! Unfortunately, a quick test with WebPageTest confirmed the matter, and also showed that the 302 redirects were not being cached, and the return visit was suffering the same 600+ 302 redirect penalty.

I’d paste the full waterfall into this post, but then you’d just be scrolling forever, and give up on what I had to say after a while. If you are curious, you can see the test results here: https://www.webpagetest.org/result/181023_JP_b9476f132f954277f10ab45938be4702/

Diagnosis

What is going on? Well, Webpack is fortunate to have a lot of sponsors, and they list many of them on their webpage. And each image was leading to a redirect:

Every request to:

https://opencollective.com/proxy/images/?

Redirected to:

https://images.opencollective.com/proxy/images/?

And the redirect was not being not cached:

cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Meaning that the same redirect occurs on subsequent visits The webpack page is open source, and digging around, I found an API call to https://opencollective.com/api/groups/webpack/backers

That returns a (really large) JSON file with all of the backers, including a URL to their image. And this was he source of the redirect.

So, we have found a top tool’s website whose 3rd party API is adding hundreds of extra requests to load the webpage.

Further Digging

We can fix this! Looking in the Opencollective Github repo, I found that the image subdomain change was made on a 9/17 commit on Open Collective. Nothing nefarious here, just a simple migration. Could this be the culprit? Luckily, with the HTTP Archive, we have historical data!

Looking at the 9/1/18 HTTPArchive, webpack.js.org has 0 redirects, but there are 649 redirects on the 9/15/2018 HTTP Archive crawl.

My hunch was that the 9/17 commit caused the issue, so this made me worry that my guess was incorrect. With over 1.2 million webpages tested in each HTTP Archive run — they cannot all be run in one day. The test for webpack.js.org was run at UNIX time 1537493496, which translates to 9/21/2018–4 days after the push. OK, I this pretty much seals the deal.

Filing a Defect — and Resolution

I filed an issue on the opencollective GitHub repo:

https://github.com/opencollective/opencollective-images/issues/10

And it was resolved last week. Now the webpack.js.org homepage again has 0 redirects, allowing all of those images to load in just one request each!

https://www.webpagetest.org/result/181108_98_7f290d38486ac8ce6505ad553c4abea9/

Conclusion:

Redirects add extra round trips to deliver content — delaying page load and lowering performance. Sometimes changes in third parties that your site depends on can lead to catastrophic changes in your sites performance, so regular testing is essential to catch these issues quickly.

Tools like HTTP Archive and WebPageTest are indispensable for helping to pinpoint issues, and should be part of your toolset to identify issues.

Originally published at dougsillars.com on November 14, 2018.