Spam Protection Ate My Newsletter!

Spam protection methods can be a huge relief to email users, relieving some of the tedium of deletion. Of course, if you’re sending a newsletter, you may wonder how many of your subscribers are really receiving it. Even the Developer Shed newsletter has to be watched to see if it’s being filtered. Let’s take a look at how spam protection can be your enemy or your friend.

With the wide range of how strict or lenient different spam filters are, it’s often unsettling to realize that you can’t tell how often your newsletter might just get junked before most people see it. You work hard on that newsletter every week, and people need to see it. Subscription email lists can be useful for reminding visitors to return to your site for new content, or inform them of news and events related to your organization or business. Especially for an e-business like Developer Shed, subscription mail is integral to the company.

The war on spam can be a war on a business’s bottom line, if you don’t keep an eye on your mailings. If newsletters don’t reach recipients, it’s a company that loses. Recently, Developer Shed found that its own newsletter had been red flagged by a spam watching group. We’ll look into this, and we’ll discuss what you can do to prevent this from happening to your mailings.

The awful part of spam is that it always finds a way to defy filters. The filters in place do help to sort a lot of spam, but it never does enough. Spammers find more and more ways to bend the system to get their mass mail into your inbox.

Unlike spammers, legitimate businesses with subscription mass mail get caught in the middle. Spam gets delivered and your mailing list doesn’t. Because most businesses don’t use shady spammer tricks, they can be more susceptible to spam protection. It’s still in your best interest to stick to legitimate practices, however, because cheating the system is unpredictable and can only work for so long.

We’ll look at the two things you must do to manage your newsletter and be fairly certain that it isn’t recognized as spam. First is checking the on-page content. This will be covered momentarily, but basically you want your newsletter to follow all standards to look as little like spam as possible. Second, you need to keep an eye on off-page elements, which Google can help to simplify greatly.

When an email server receives an inbound message, in most cases it will run a spam check on it. A program scans your message for certain details and keywords. Your mail starts with a score of zero. For each negative detail or keyword, the program adds points to your score. If the score reaches a certain level (specific for the program’s settings), it will not be delivered.

On-page optimization is critical because if performed properly, it can keep this score low. One of the best things you can optimize is your mail’s subject line. As an example, let’s take a look at the subject line of a recent Developer Shed newsletter:

Developer Shed Weekly Update for 2005-07-19

This subject makes it completely transparent what the message is going to be. Use a word to indicate the mail is a newsletter, such as “news” or “newsletter” or “Update.” Include the date or issue number in the subject. It also doesn’t hurt to include the publishing frequency, “weekly” in this case.

Also keep your subject clear of anything that would make somebody scanning think it was spam. Words like “XXX,” “free,” “penis,” “loan,”  “deals,” etc. can quickly rack up points on spam filters. That’s why spammers intentionally misspell words or use other characters (like *) in the middle of words. These tricks should always be avoided, as they downright say “this message is junk.” What fools spam filters today will be penalized harshly tomorrow. Don’t use any tricks.

On the same note, if you distribute a legitimate newsletter with a title like, “Weekly Guide to Cheap Real Estate,” rename it. Your subscribers may want it, but it could look quite bad to a filtering program.

Your content is also key here. Not only does real substance help pass through spam filters, but it helps make your readers value your newsletter. If they enjoy your newsletter updates and for whatever reason fail to receive one, hopefully your subscribers will complain about it so you can troubleshoot.

This should be painfully obvious to anyone, but do not type anything in all caps. Your subject line should be mixed case. Your content should also be mixed case. Follow proper grammar rules in this sense. Emails that are all caps are often junk mail.

This is quite important. Always provide an opt-out link or directions within the newsletter. This is both a matter of legality and sensibility. Use words like “unsubscribe” and “opt out” in this part of the email. Spam often uses the word “remove,” so it may actually hurt you more than help depending on the filtering program. Make sure that when a person sends an unsubscribe request it is handled quickly.

Under no circumstances should your newsletter have an attachment. This puts up red flags for readers and web servers. It looks suspicious for any mass mail to be sending a package. It also eats up valuable inbox space and takes extra time for you to send. If you need to include images or a file, host them on your webpage server and link to them.

To get through spam filters, you should also send from a reputable email domain and never dupe the ‘from’ address. Especially with technology like Sender ID coming, do not get caught with a forged email header. Sending from your website’s own domain is always the best option.

There are plenty of guidelines and common sense rules to follow. In case you miss something or do not optimize enough, there are automated tools such as SpamCheck (visit it online). You must send a copy of your newsletter to the email address that they specify, with a modification made to the subject line. By adding TEST to the beginning of the subject, the program recognizes it as a request for service rather than spam.

Within minutes, you will receive an email that details things that could be improved on your newsletter. It will also give you the score that your newsletter will receive using its spam filtering program.

The Developer Shed newsletter scored very well. Below is an explanation of the scores and our detailed results.

Your TOTAL SPAM SCORE for your e-mail was 0.4.

How To Interpret Your Score:
————————————————–
The higher the score, the more likely the e-mail
will be considered spam by ISPs, which means the
more likely it is to get filtered.  Levels…

0.0 – 4.5 – nice and clean, no problems except tiny
ones below; no action required

4.6 – 7.0 – the strictest may object; clean up the
easy-to-find issues (below)

7.1 – 10.0 – getting into dangerous territory; clean
up any big issues and the easy-to-find smaller ones

10.1 – 13.0  likely overISPlimits; requires good
review and cleaning up

13.1+ major problems; overhaul needed –
systematically clean, point by point and then
re-test (this may require two or three checks).
————————————————–

In order of importance, here are the mistakes that we found
in your e-mail…

(0.1 points) BODY: HTML font color is unknown to us
(0.1 points) An exceedingly large amount of HTML coding is often used to disguise common spam phrases. Consider reworking slightly.
(0.1 points) BODY: HTML font color is blue
(0.1 points) BODY: HTML has a big font
(0.0 points) BODY: HTML included in message
(0.0 points) Asks you to click below

Eliminate mistakes by removing any common “spammer habits” from your e-mail. You’ll find it’s a useful style check at the same time. We all tend to repeat some words too often. The net result is a cleaner (no-spam-triggers) e-mail, that will be opened and read by more people.

If you send the copy of your email with your regular mail server, your score will likely be better than if you just forward it from a client like Outlook or Thunderbird. The spam filer takes off some points if it thinks you are trying to dupe it with a mail program. Of course, it details how many points that adds to your score so you can subtract those points if need be.

There are spam watchers outside of those that watch what comes in on mail servers. Some people have individual clients or use spam databases to help filter mail. These resources are not foolproof.

The easiest tool to check for this sort of a problem is Google. First, choose a newletter; you don’t want to pick the most recent newsletter, since it may take a couple days for an online spam resource to post your mail online. Also, don’t count on search engines having gotten a chance to index the page if the newsletter is very recent.

Once in your friendly search engine, type the subject line of a newsletter you mailed in quotation marks (“Developer Shed Weekly Update for 2007-07-19” or “Developer Shed Weekly Update for”). If this returns nothing unexpected, that’s good. But try again. Enter a phrase from inside the newsletter in quotation marks. Enter the email address which sends your newsletter. Hopefully, you’ve gotten nothing but the mirror of your own newsletter (which you should be posting on your website).

This is how we found the red flag on Developer Shed’s newsletter. It was erroneously included in one of these online spam catalogs, Spam Trace. The Spam Trace service includes an email blocker the prevents POP email addresses from recieving mails in its database.

So let’s see how to handle the situation if your newsletter turns up in these databases. Any reputable spam tracker will have a contact address, though you may need to go hunting for it. Once you have located it, send a polite email explaining the situation in detail.

As an example, here is what Developer Shed’s CTO mailed to Spam Trace:

Hello,

It has been reported to us that you have marked our weekly newsletter as spam on your site.  Here is a URL:

http://www.spamtrace.info/email/3364/

Our weekly newsletter is anOPT-IN service, and we offer a clear opt-out at the bottom of each and every newsletter.

Please look into this, and remove us from your spam list.  You can contact me directly with any questions.

Rich Smith,CTO
Developer Shed, Inc.
www.developershed.com

Two days later, we received this response:

Rich,

Thanks for bringing this to my attention.

I apologise for this, the process of gathering emails is automatic and the amount through each day is large.  We do have processes in place to stop things like this unfortunately this fell through the cracks.

I’ve checked the full database and your newsletter had 9 entries, I have deleted them all and put code in place so your site should no longer but put in our database.

Regards,
Mike

Whether it was a cataloging mistake or somebody had submitted our newsletter to the company, we were being penalized for our regular mailings despite them following all the rules. Thankfully, Spam Trace was very effective, and by the time we received that email none of the Developer Shed newsletters remained on their server.

The popularity of these online spam databases is growing as spam becomes more prevalent. One recent proposal (visit it online) calls for uniting Canadain ISPs under one spam database to help prevent people from recieving junk and tracking down illegal scams that are mass mailed.

You don’t want to be part of this group. Databases such as this exist largely to help law enforcement agencies prosecute senders of the archived mail. If your mail is properly optimized with an unsubscribe link, you should be in the clear. But to avoid any future hassle, keep up-to-date on your newsletter’s whereabouts anyway.

The United States’ FTC has a similar program that it is starting, which you can visit you can also visit if you click here. It has no stated plans for a spam database, but it’s probably not unthought-of. Regardless, not providing proper on-page optimization for opting out or clearly looking like a reputable newsletter may compel a spam activist to turn your mail in. This could result in some undesirable legal situations down the line.

When all these spam services come together and do what they are supposed to, it can be a great thing. It doesn’t just help people reading email, it helps people sending newsletters. Granted you spend the time to do a check-up on your letter, it will arrive in cleaner inboxes where it can receive more attention. When you don’t have undesirable, impersonal email to compete with, it’s always good to help attract some views.

Always keep in mind that using RSS and copying your newsletter to a dedicated page on your website offer more ways to update your readers without losing them. Developer Shed has RSS feeds for every one of the sites on the network, which people can post on their own pages or track on their own. These feeds can bring you the attention that mass mailings cannot, if your newsletter gets overlooked in piles of spam. Also, having an HTML page of past publications set up is also helpful. Not only can it keep people up to date, but it also can get spidered by search engines.

It may seem obvious, but never rely on cheap tricks to get more readers. One mistake people make is to sign up people to their list automatically. Even if your unsubscribe function works perfectly, your newsletter loses all respect and value when it is sent unsolicited.

Finally, there may be good revenue in mailing advertisements to your mailing list. This should not be done outside of your newsletter, unless it is something subscribers consciously opt into. Placing an add in the regular mailing doesn’t hurt, but becoming a source of “subscribed spam” is only going to hurt your reputation and subscribers.

The war on spam can be viscous to anyone with a mailing list. Clearly, if you work it just right, spam filters can be your friend. If you ignore the details and just try to get content out there, you may suffer for it.

Google+ Comments

Google+ Comments