More about email deliverability

Sometime ago I wrote a post about how to send emails without getting flagged as SPAM. Unfortunately I had missed out on a few things.

rDNS

One of the most important things in email deliverability is rDNS (also known as the PTR record). rDNS stands for “reverse” DNS. Here is an example how rDNS works:

• A mail header says that the sender is abc@abc.com and it was sent from 111.222.333.444
• The receiving mail server verifies if 111.222.333.4444 really points to abc.com by a rDNS lookup

Mail servers like AOL and Google are very very particular about rDNS i.e – they would check for reverse DNS entries for each mail received by them. Your important newsletter is almost sure to make it to the bulk email folder if you haven’t setup the reverse DNS entry for your domain.
Now if you are hosting your nice web-application on Amazon EC2, reverse DNS won’t work for you, because Amazon won’t set your reverse lookup. You will have to use a third-party email service like authsmtp or fastmail to send out mass emails with maximum deliverability.

Domain Keys Identified Mail

Domain Keys Identified Mail is a method for email authentication (as the DKIM website says). Basically DKIM allows the sender of an email to sign the email using public key cryptography. Prominent email services like Yahoo, Gmail and Fastmail implement DKIM. This is how it works in a nutshell:

• The sender of the email adds a header-field named “DKIM-Signature” which contains a digital signature of the contents of the header and body of the email message
• The receiving SMTP server does a DNS lookup and gets the public key for the domain.
• It uses the public key to decrypt the message

You can use Javamail with DKIM for sending out that important newsletter from your web application.

Sender policy Framework

SPF is a special format DNS record which specifies which machines can send emails for that domain.

• For example the owner of abc.com can determine which hosts are allowed to send emails whose sender email address ends in @abc.com
• Receivers who check SPF can reject messages from unauthorized hosts before receiving the message body

And here’s what the SPF record may look like:

abc.com TXT “v=spf1 ip4:111.222.333.444 –all”


White lists, Black lists

• Blacklists are lists of domain names which are known to send SPAM emails. Basically they are the lists of known offenders.
• Whitelists are the opposite – lists by which an ISP allows someone to bypass spam filters when sending emails to its subscribers

So …. Get Registered

MxToolBox and ReturnPath are both good options

How to avoid getting “Flagged as Spam” while sending legitimate emails

The menace called SPAM is a double edged sword. On one hand we have to constantly fight to keep SPAM from reaching our mailboxes. On the other hand we have to be careful so that the legitimate emails that we send don’t end up being caught in the net of anti-spam software. Take the example of sending automated emails to people who sign up for the beta version of an exciting product that we are building. These emails are not unsolicited. The ultimate control for the delivery of such emails reside with the servers receiving the emails and the anti-spam policies and software they implement. However following the guidelines below will reduce the chances of getting flagged by anti-spam software drastically.

I. Don’t spoof your identity.

Be accurate in who you are and from where you are sending the email. It’s always good to send your emails from your own servers, using your own domain name. For example if you have an application running on a machine in the domain example.com and you are sending emails from a mail server in that domain, it’s preferable that the senders address be someone@example.com. If you try to hide your source and destination you’ll look like spam. Don’t’ add unnecessary headers to the emails.
The email with the following header went to the junk email folder.


Received: by *ip-xxx-xxx-171-173.ip.xxxxserver.net* (Postfix, from userid 99)
id 7C95F298101; Wed, 9 Jul 2008 05:16:24 \-0700 (MST)
Received: from ip.secureserver.net (ip-xxx-xxx-171-173.ip.xxxxserver.net [127.0.0.1])
by ip-xxx-xxx-171-173.ip.secureserver.net (Postfix) with ESMTP id 6385E2980F1
...
Date: Wed, 9 Jul 2008 05:16:23 \-0700
From: *admin@somedomain.com*

The domain names of the “Received: ” and “From:” fields don’t match.

II. Genuine domain names

Use a domain name which is identified by a verifiable IP address. For this reason, it is very important to have rDNS (Reverse DNS) entry, also known as a PTR record for the server from which you are sending emails. Most anti-spam software reject emails sent from servers that don’t have an rDNS entry.

The email with the following header went to the junk email folder.

Received: from *unknown* (HELO ip-xxx-xxx-171-173.ip.xxxxserver.net) (208.109.171.173)
by k2smtpout06-01.prod.xxxx.xxxxserver.net (xx.xx.189.102) with ESMTP; 08 Jul 2008 06:34:42 \-0000

“unknown” in the “Received” header means the receiving server could not determine the identity of the server sending the email which generally turns out to be the lack of an rDNS entry for the sending server.

III. Send well constructed emails

Emails with missing mime sections, invalid or missing message-ids, invalid or missing date headers, or subject etc., are frequently signs of spam.

IV. Encodings

Avoid needless encodings and charsets in the emails. Don’t use base-64 encoding unless you really need to.

Consider the Subject field in the header below:
Subject: =?iso-8859-2?B?U1BBTTpSZWdhaW4geW91ciBuYXR1cmFsIHdlbA==?=
=?iso-8859-2?B?bG5lc3M=?=
Content-Type: text/plain;
charset="iso-8859-2"

The character set in this email is ISO-8859-2 which is the unicode encoding that a lot of eastern European countries like Hungary, Poland et al. use. This message ended up in my junk mail folder.

The following is a part of the header which ended up in my junk mail:

X-OriginalArrivalTime: 13 Jun 2008 13:52:08.0648 (UTC) FILETIME=[AC00D480:01C8CD5C]
\--Apple-Mail-16-982198482
Content-Disposition: inline;
filename="Picture 6.png"
Content-Type: image/png;
x-mac-hide-extension=yes;
x-unix-mode=0644;
name="Picture 6.png"
Content-Transfer-Encoding: base64


V. HTML emails

Malformed HTML

If you’re using HTML emails then the least you can do is to make sure that the HTML is valid. Unbalanced and invalid tags are bound to flag an email as spam.

Invisible text in HTML

If you’re using HTML emails, do not use invisible text within those emails. Make sure your text colors and sizes are distinct enough and large enough to read. Invisible text (e.g – text color is the same as background color) is often identified as a sign of spam.

Consider the following logo in a HTML email:

Welcome Everyone

The following is the source for this:
< p style="color:#4d4d4d;font-family:Arial,Helvetica,Verdana,sans-serif;font-size:24px;font-weight:bold;margin:0;padding:5px 0 0 5px;">Welcome<span style="color:#808080;"> Everyone</span></p>

This is good and valid. However the following is not:

Since it’s some text disguised as an image.

VI. Keep it simple

Do not use cute spellings, Don’t space out your words, don’t put str@nge l3tters 0r characters into your emails. You are bound to look as spam if you do.

For example some people emphasize/stylize the text by writing:
L E G I T I M A T E .
Text like such will make the email likely to get caught in spam filtering.

If you found this post helpful, there is some more stuff on the same topic in a later post of mine over here