I am not the world's greatest detective.

To be honest, I'm not even the fourth best, but I didn't let that stop me from solving the Mystery of the Busted Varnish Cache!

We were minding our own business, doing performance reports, checking data, etc when we noticed that we had several sites where it seemed like the home page wasn't being cached.

How did we know this? By the abysmal load times! 10+ seconds?! What?!

We needed more data.

Backstory

We host a 400+ sub-site WordPress Multisite network with Pantheon hosting.

We inherited it after years of spotty maintenance.

There are so many things to refactor, that one code cleaner thinger estimated it would take us 34 years to clean it all up. Hahahaha but really, I'm sobbing here.

So, long story short, we rely a lot on caching to make the visitor experience less crap while we get our code in order.

This is the sort of compromise you sometimes have to make IRL.

Troubleshooting

First up, we visited the website via Chrome. It was fine but slow. However, the main URL immediately performed a redirect and /en was appended to the URL.

Ahhhh, it was a bilingual WordPress site using the Polylang Pro plugin to switch between French and English. (Bonjour! Je Suis Canadienne, eh? I can also tell you I went to the library with a grapefruit en Français and other Year 1 phrases.) To make the switch, the homepage redirects to a page with either /en or /fr in the URL.

With this in mind, we used curl from the command line to see what was happening with headers etc.

curl -I https://www.domainname.ca

Hm. The information we got back was as expected:

cache-control: public, max-age=86400
via: 1.1 varnish
age: 53
x-served-by: cache-mdw17340-MDW, cache-sea1027-SEA
x-cache-hits: 0, 1
x-cache: Miss from cloudfront

The cache-control was set to 24 hours, that's cool. (Pantheon has a plugin that will refresh the cache of a single page if you update it. Long cache TTLs are your friend, if you can use them.)

We have an age, which means we have a working cache.

Huh. So... Varnish was getting hit but Cloudfront wasn't? A quick check shows that our Cloudfront isn't configured to cache so that explains that.

And yet... why are we getting such slow load times if we're hitting the Varnish cache?

Let's try the redirected URL:

curl -I https://www.domainname.ca/en

Oh-ho! This gave us some unexpected information:

set-cookie: pll_language=en; expires=Fri, 31-Jul-2020 19:37:04 GMT; Max-Age=31536000; path=/; domain=.xxxxxxxxxxxxxxx.io
cache-control: public, max-age=86400
via: 1.1 varnish
age: 0
x-served-by: cache-mdw17359-MDW, cache-sea1030-SEA
x-cache: MISS, MISS
x-cache-hits: 0, 0
x-cache: Miss from cloudfront

Huh. That URL is definitely not caching anything. (Age is zero, x-cache is a double MISS, and x-cache-hits are zero.)

But wait! What the hairy heck is that set-cookie header doing there? That wasn't on the home page...

Solution

Pantheon has a page all about debugging cookies and their global CDN which explained what was going on.

If your output looks like the following (i.e. an Age of 0) after more than one request, check if a cookie is being set (with the Set-Cookie header). Setting cookies will prevent the CDN from caching that page

Bing bing bing! It looks like our Polylang Pro plugin was setting a cookie for language preference using the set-cookie header!

If we weren't running 100+ sites with this plugin to handle translations we might have considered swapping it out for a different plugin.

Thankfully, it turns out that GDPR meant Polylang Pro had developed a way to prevent the pll_language cookie from setting.

Our wp-config.php file was updated to include this change and bam, now our multilingual homepages are caching.

Of course, the trade-off is that if you return to a site it won't remember your language preference but that seems like a small sacrifice to make for the site loading 8 seconds faster.

¯\_(ツ)_/¯

Base Photo by Joanna Kosinska on Unsplash