After in-depth analysis of SendGrid system, I came to the following conclusions:
- Their "open" event is not accurate, as it can be trigger by either a BOT or a human
- They know this (and have built their analytics dashboard, knowingly or unknowingly, so that it doesn't reflect this issue)
On the above screenshot, you can see that 46 emails are being opened within 100 seconds exactly. There were 96 recipients in total, so that'd mean about 50% opened the email right away?
That's odd, isn't?
But let's do more testing:
On this batch, we got even better numbers!
66 recipients, 52 of them opened the email within 40 seconds. 🎉
Our marketing team must be awesome, and our product terrific for such quick enthusiasm!
Well, except the "click" rate is ... 1%. Odd, isn't?
But how does that "open" event works, by the way?
It's simple. SendGrid adds a "pixel" in the email, basically an image which points to a script. Whenever that script is being downloaded, it means the email has been opened. Simple!
This might not be 100% accurate, but you get the big picture.
👉 SendGrid doesn't know that you've opened the email by actually clicking on it, they only know that it has been opened.
And what else could open our emails? 🤔
Well, you guessed it (probably): SPAM checkers, and any other "AI" that would read the email content (like Gmail, to know whether to store it into your "Promotion" inbox and all of that).
In conclusion, SendGrid cannot make the difference between an email opened by an actual human, or by a bot!
And that would be fine, it they weren't trying to hide it. (or at least, make it obvious that this information is not 100% accurate!)
Companies use KPIs to run a business, and sometimes, those KPIs are "how many people are reading our newsletter?". Well, when you make strategic decisions like that based on deceptive information, it can hurt.
Following this analysis, I've contacted the "Trust & safety" team at SendGrid with a few advices on how to be more transparent about this fact, and decided to write a public article about it.
I believe companies should put transparency over growth.
Don't just hide your problems, but fix it instead, or at least tell us about it.
N.B: I bet other "Email delivery providers" have the exact same issue, and are deceptive, too.
P.S: I hope nobody got fired "for doing a bad job" because their email campaign had a 1% clicks out of 80% reads. But, murphy law, right?
Top comments (2)
It's been known for a while that pixel tracking isn't the most effective method for analytics, and I'm not surprised that middleware systems like spam filters are creating false positives.
It's likely more effective to have analytics around your outbound links from the email service, that might not tell you how many opened but the click statistics would offer more insights.
Indeed they are deceptive - they're incentivized to try to be seen as a powerful tool, so they're fine with e.g. managers seeing inflated open rates on reports without realizing the inaccuracy.
Are you "Tracking Unique Opens and Clicks with the Event Webhook"?
Thanks for reporting on this.