Sometimes when a test fails, my first reaction will be to instantly re-run our continuous integration suite, without investing the time to evaluate the failing spec. This is a habit I want to overcome. To justify myself here, I do this because I will eventually acknowledge the problem when it happens repetitively, enough to annoy me. We're often preoccupied with a different task, and slowing down to thoroughly look at a random unrelated failing spec is hard to do.
Recently, one or two tests would fail randomly in our CI suite and I would see a message similar to this:
For those of you who aren't familiar with RSpec, what it is saying is that it can't find Vallie O'Keefe in the response but it is clearly displayed there. I instantly rerun the spec and 90% of the time it would pass. But the problem is it would come back and haunt randomly here and there.
After about 2 weeks of this random failing spec (which had been appearing throughout our request specs, and only appeared to happen to me for some reason), I said to myself "Alright, what the hell is going on?". I ran the possible failing spec a few times locally and it seems to pass. I immediately realized this was a rare edge-case that I will hit if I repeatedly if I ran the spec 50+ times. So, I whipped up a script and I patiently sat and stared at it.
On the 20th attempt, the spec finally failed.
It turns out that whenever the Faker gem, which randomly generates test data, created an output that contains an apostrophe, the apostrophe was getting encoded/escaped in HTML. When this happens, Vallie O'Keefe won't get matched with Vallie O'Keefe. What baffled me is that the response's content is unescaped when it is rendered on the CI suite. To remedy this, I had to escape what I wanted to match the response with. Below, for an example:
expect(response.body).to eq CGI.escapeHTML(@thing.name)
I wish I knew this before we start writing request specs. BUT MORE IMPORTANTLY, I wish I slowed down and investigated it so that I could take precautionary measure for all the future request specs. Instead, I now have to manually spot failing request specs on CI and update as I go.
Top comments (1)