GMail Filters Flagging Legitimate EMails as “Dangerous”
For more than six months now, since Novembe 2019, GMail has repeatedly been blocking legitimate EMails from us, and consistently ignoring all of our requests that they fix the issue. We first became aware of the issue when several of our customers informed us that they hadn't received EMails they were expecting from us, with invoices for the services we provide to them. After some further checking, it turned out that the messages had ended up in the "Spam" folder of the recipient's GMail accounts, and GMail had flagged them as "Dangerous." Normally, marking the messages as "Safe" and/or adding the sender's address to the contact list will solve the issue - though in this case, the EMails from us were still filtered into the spam folder even after following those steps.
Since the usual solutions didn't work, we ended up doing some testing to try to determine why the messages were being filtered - which was more made difficult to GMail's lack of transparency over how their filters work. Granted, and in context, most mail providers don't publicly provide full details how their spam filters are configured (we certainly don't), because that information would make it easier for spammers to get around those filters; but with most hosts & spam filtering solutions, the recipient of the message can at least view the raw headers of the message to see details of how it was classified by the spam filters. Unfortunately, though, GMail lacks even that degree of transparency - so the only way to test was via brute-force trial-and-error: namely, editing the attachment that GMail considered "dangerous" to remove one line at time, the resend, ad infnitum, until we identified the specific text that was triggering the filter.
Surprisingly, it turned out that the messages were being filtered for the sole reason that they contained a PDF attachment, which in turn contained text content with a ".com" domain name. As should be obvious, this practically guarantees an absurdly high rate of "false positives" - AKA, legitimates messages erroneously filtered as spam or malicious. For ourselves, as a provider of hosting & domain registration service - and given that ".com" is the most commonly-used type of domain name in the world, that means most of the invoices we send to customers who use GMail will be blocked by their filters. Or it would, except that on top of that, our formally registered company name also includes our (.com) domain name - so EVERY invoice we send contains a .com domain name, because they contain our company's name.
Some additional details that we discovered during the testing:
- It appears that they've manually "whitelisted" their own domains, as the filter conveniently ignores PDF attachments containg "gmail.com" and "google.com".
- Doesn't matter if the domain name is actually registered or not, a message with an attachment containing "someinvaliddomainthatisntevenregistered.com" was flagged as "Dangerous"
- Doesn't matter if the domain name isn't even valid & would be impossible to register - a message with an attachment containing "someinvaliddomainthatisntevenr&egistered.com" was flagged as "Dangerous"
Not only does the filter have an inexcusably high rate of error/false positives, it's ALSO appears to have an etremely high rate of false negatives (spam/malicious EMails that the filter deems safe) - meaning that not only does it block significant amounts of legitimate messages, but it won't even effectively work with the majority of spam/malicious messages that it's targeted at. The reason being that it's absurdly easy to get EMails with malicious links past that filter, simply by using any type of domain name other than .com or .net - which spammers have already been doing for years now, as many of the other types of domains are cheaper & have much less oversight from the registries that control them. Or there's another well-established trick that spammers have been using years: obfuscate the actual destination of the link by hiding it behind a URL shortener such as bit.ly... or Google's own "goo.gl" service (and yes, the filter does also ignore messages containing goo.gl URLs, even if they redirect to known malicious URLs). It's a textbook example of IT "security theatre" at its worst: it causes significant problems for legitimate uses, while not even being particularly effective at addressing the actual problem it's designed to deal with.
We have repeatedly attempted to contact GMail & request that they fix that issue (most recently via Twitter) - but unfortunately, they've ignored all those requests & haven't even bothered to respond; and have evidently failed to do anything about the problem, given that it's still occurring at the time of this writing. So, unfortunately, we now have to recommend that all customers of ours avoid using GMail for any important communication. And that's probably a good idea for all users of GMail, not just our customers - given that GMail clearly can no longer be trusted to reliably deliver legitimate EMails, and given their apparent lack of concern for that issue.
