I’ve spent much of today talking to various people about IP reputation and bulk foldering. It’s an interesting topic, and one that has changed quite a bit in the past few months. Here are a few of the things I said on the topic.

Generally IPs that the ISP has not seen traffic from before starts out with a slight negative reputation. If you think about all the new IPs that an ISP will see mail from on a daily basis, 99 out of 100 of those will be bot infected windows boxes. So they’re going to treat that mail very suspiciously. And, in the grand scheme of things, that mail is going to be spam a lot more than it’s not going to be spam.

Some ISPs put mail in the inbox and bulk foldering during the whitelisting process. Basically they’re looking to see if your recipients care enough about your mail to look for it in the bulk folder. This then feeds back to create the reputation of the IP address. There is another fairly major ISP that told me that when they’re seeing erratic data for an particular sender they will put some mail in bulk and some mail in the inbox and let the recipients tell the system which is more correct.

That’s what happens while you’re establishing a reputation on an IP. Once there is some history on the IP, things get a little different. At that point, IP reputation becomes unimportant in terms of bulk foldering. The ISP knows an IP has a certain level of reputation, and *all* their mail has that level of reputation. So bulk foldering is more related to content and reputation of the domains and URLs in the message.

The other reason IP reputation isn’t trumping domain / content reputation as much as it did in the past is that spammers stomped all over that. Affiliates, snowshoers, botnets, all those methods of sending spam made IP reputation less important and the ISPs had to find new ways to determine spam / not spam.

So if you’re seeing a lot of bulk foldering of mail, it’s unlikely there’s anything IP reputation based to do. Instead of worrying about IP reputation, focus instead on the content of the mail and see what you may need to do to improve the reputation of the domains and URLs (or landing pages) in the emails. While the content may not appear that different, the mere mention of “domain.com” where domain.com is seen in a lot of spam can trigger bulking.

I use a pretty standard mail filtering setup – a fairly vanilla SpamAssassin setup on the front end, combined with naive bayesian content filters in my mail client. So I don’t reject any mail, it just ends up in one of my inboxes or a junk folder. And I have a mix of normal consumer mail – facebook, twitter, lots of commercial newsletters, mail from friends and colleagues and spam. (As well as that I have a lot of high traffic industry mailing lists, but overall it’s a fairly normal mix.)

My bayesian filter gets trained mostly by me hitting “this is spam” when spam makes it to my inbox. If I’m expecting an email “immediately” – something like a mailing list COI confirmation or email as part of buying something online – I’ll check my spam filter and move the mail to my inbox in the rare case it ended up there. Other than that I let it and spamassassin chug along with no tweaking.

I’m starting a data analysis project, based on my own inboxes, and as part of that I’m using some tools to look for false positives in my junk folders, and manually fixing anything that’s misclassified. I’ve been doing this for a couple of hours now, and I’ve found some interesting things.

1. Simple content filters work remarkably well out of the box, at least for my mail stream. Spectacularly well. There’s very little in the way of false positives. Very, very little.
2. Of those false positives there’s nothing I’d have been bothered about. It’s generic, unexciting junk mail.
3. Most of the systemic false positives seem to be correlated with the senders doing something bad. Heathrow Express, for instance, sent me mail every two weeks or so since I’d signed up. Then for no obvious reason they stopped sending for three months, then started sending again. Every mail they sent after that pause ended up in the junk folder, and I never missed them.
4. I get regular newsletters from ThinkGeek. Every one of those goes to the inbox. I occasionally get mails from them about my account (“you’ve got 420 geek points left”) that are kinda transactional, but not something I expect to see – and they all end up in the junk folder. Several other senders do the same thing, and get the same result.
5. Several companies have used tagged addresses to send me newsletters for a while, and also used them to send unsolicited facebook invites (to the tagged address, from facebook servers). The regular newsletters all go to the inbox, while the facebook invites all go to the junk folder. “Legitimate” facebook mail, meanwhile, keeps going to the inbox.
6. Apple send me a lot of newsletters – I’m a Mac and iPhone developer, I get their consumer newsletters, transactional stuff from our local store – lots and lots of newsletters. They all made it to the inbox except for one. The one that ended up in the junk folder was a one-off about recycling, and it wasn’t up to their usual design standards – it had ugly big green “call to action” headlines in it, very different to their usual clean design.
7. Just one sender hit the junk folder every time. The distinctive thing about their messages (apart from them not being something I missed) was that the plain text part of them was dreadful, just a bad lynx dump of the html section. Even a “No plain text for you! Go to this link!” would have been better.

I was surprised at how effective this simple content-based filtering setup had worked with little tuning other than hitting the this-is-spam button – both in it’s accuracy at removing spam while keeping a very low false positive rate, but also how well the false positives matched my judgement of “Meh. This mail isn’t interesting.”.

We spend a lot of time talking about the things you should do to make the mail relevant to the recipient – compelling content, consistent, predictable delivery schedules, clear consistent branding, use of a single consistent mail stream to communicate with a recipient rather than several different streams. Until I went through this exercise it wasn’t clear to me how much of an effect those things also have on fairly simple recipient-trained filters.

Cloudmark has an interesting blog post pointing out some of the challenges of signing mail with DKIM in a large company with a diverse mail system.

I recently received a survey from my Congressional Representative. She wanted to know what I wanted her to focus on in the coming year. I decided to go ahead and answer the survey, as I have some rather strong opinions on some of the stuff happening in Congress these days.

The email itself was pretty unremarkable, although quite well done. I was as much interested in answering the survey because it’s one of the few emails I’ve seen with an embedded survey.

Screenshot of the email survey

I was a little unnerved by the note at the bottom, under the submit button. “By clicking submit you will automatically be opted into receiving regular updates from my e-newsletter.” It’s not necessarily that I mind being opt-ed in, but I get mail from her often enough that I’m pretty sure that I’m already opted in. But, OK, I clicked submit.

I noticed this was being hosted on a website called address-verify.com. OK, so it’s been outsourced to a 3rd party. But what I saw on the website caused me to recoil in horror, both as a email recipient and as an email expert. The “subscription page” doesn’t look like it was intended to be shown to the general public. (Click on the image to see the full form)

This appears to be more information than any recipient would give

My first reaction was “You can not seriously expect me to give you all of that data! And why are you asking for my email address, you have it! You sent me email!” Then I started looking at the form a little harder and it actually looks like an internal form used to track constituent contact and not one that was supposed to be exposed to recipients.

I’m sure this is all valuable information for my rep to have, but I’m not going to spend a lot of time filling out the long form. It’s too much to hand over.

I’m disappointed. I actually wanted to give my Rep. the information she was asking for. And I happily answered the survey in the email. I really appreciated the initial email and the subscription notice on the email. It seemed like a well put together campaign and I am happy to give her my feedback. I’m not sure anyone at the office actually looked at the landing page before the email was deployed.

Unfortunately, this is not as uncommon as it should be. Sometimes senders don’t pay attention to landing pages and actually check mails before deploying them, particularly when they’re sending something new. In this case, they lost the chance to engage more with me. And I lost the chance to engage more with them as I just don’t want to spend 20 minutes filling out pages and pages of a survey.

This is a great story from Trey Harris about a real email delivery issue from the mid 1990s.

Here’s a problem that sounded impossible…  I almost regret posting the story to a wide audience, because it makes a great tale over drinks at a conference.   The story is slightly altered in order to protect the guilty, elide over irrelevant and boring details, and generally make the whole thing more entertaining.

I was working in a job running the campus email system some years ago when I got a call from the chairman of the statistics department.

“We’re having a problem sending email out of the department.”

“What’s the problem?” I asked.

“We can’t send mail more than 500 miles,” the chairman explained.

I choked on my latte.  “Come again?”

“We can’t send mail farther than 500 miles from here,” he repeated.  “A little bit more, actually.  Call it 520 miles.  But no farther.”

“Um… Email really doesn’t work that way, generally,” I said, trying to keep panic out of my voice.  One doesn’t display panic when speaking to a department chairman, even of a relatively impoverished department like statistics.  “What makes you think you can’t send mail more than 500 miles?”

“It’s not what I think,” the chairman replied testily.  “You see, when we first noticed this happening, a few days ago–”

“You waited a few DAYS?” I interrupted, a tremor tinging my voice.  “And you couldn’t send email this whole time?”

“We could send email.  Just not more than–”

“–500 miles, yes,” I finished for him, “I got that.  But why didn’t you call earlier?”

“Well, we hadn’t collected enough data to be sure of what was going on until just now.”  Right.  This is the chairman of *statistics*. “Anyway, I asked one of the geostatisticians to look into it–”

“Geostatisticians…”

“–yes, and she’s produced a map showing the radius within which we can send email to be slightly more than 500 miles.  There are a number of destinations within that radius that we can’t reach, either, or reach sporadically, but we can never email farther than this radius.”

“I see,” I said, and put my head in my hands.  “When did this start?  A few days ago, you said, but did anything change in your systems at that time?”

“Well, the consultant came in and patched our server and rebooted it. But I called him, and he said he didn’t touch the mail system.”

“Okay, let me take a look, and I’ll call you back,” I said, scarcely believing that I was playing along.  It wasn’t April Fool’s Day.  I tried to remember if someone owed me a practical joke.

I logged into their department’s server, and sent a few test mails.  This was in the Research Triangle of North Carolina, and a test mail to my own account was delivered without a hitch.  Ditto for one sent to Richmond, and Atlanta, and Washington.  Another to Princeton (400 miles) worked.

But then I tried to send an email to Memphis (600 miles).  It failed. Boston, failed.  Detroit, failed.  I got out my address book and started trying to narrow this down.  New York (420 miles) worked, but Providence
(580 miles) failed.

I was beginning to wonder if I had lost my sanity.  I tried emailing a friend who lived in North Carolina, but whose ISP was in Seattle. Thankfully, it failed.  If the problem had had to do with the geography of the human recipient and not his mail server, I think I would have broken down in tears.

Having established that–unbelievably–the problem as reported was true, and repeatable, I took a look at the sendmail.cf file.  It looked fairly normal.  In fact, it looked familiar.

I diffed it against the sendmail.cf in my home directory.  It hadn’t been altered–it was a sendmail.cf I had written.  And I was fairly certain I hadn’t enabled the “FAIL_MAIL_OVER_500_MILES” option.  At a loss, I telnetted into the SMTP port.  The server happily responded with a SunOS sendmail banner.

Wait a minute… a SunOS sendmail banner?  At the time, Sun was still shipping Sendmail 5 with its operating system, even though Sendmail 8 was fairly mature.  Being a good system administrator, I had standardized on Sendmail 8.  And also being a good system administrator, I had written a sendmail.cf that used the nice long self-documenting option and variable names available in Sendmail 8 rather than the cryptic punctuation-mark codes that had been used in Sendmail 5.

The pieces fell into place, all at once, and I again choked on the dregs of my now-cold latte.  When the consultant had “patched the server,” he had apparently upgraded the version of SunOS, and in so doing downgraded Sendmail.  The upgrade helpfully left the sendmail.cf alone, even though it was now the wrong version.

It so happens that Sendmail 5–at least, the version that Sun shipped, which had some tweaks–could deal with the Sendmail 8 sendmail.cf, as most of the rules had at that point remained unaltered.  But the new long configuration options–those it saw as junk, and skipped.  And the sendmail binary had no defaults compiled in for most of these, so, finding no suitable settings in the sendmail.cf file, they were set to zero.

One of the settings that was set to zero was the timeout to connect to the remote SMTP server.  Some experimentation established that on this particular machine with its typical load, a zero timeout would abort a connect call in slightly over three milliseconds.

An odd feature of our campus network at the time was that it was 100% switched.  An outgoing packet wouldn’t incur a router delay until hitting the POP and reaching a router on the far side.  So time to connect to a lightly-loaded remote host on a nearby network would actually largely be governed by the speed of light distance to the destination rather than by incidental router delays.

Feeling slightly giddy, I typed into my shell:

$ units
1311 units, 63 prefixes
You have: 3 millilightseconds
You want: miles
* 558.84719
/ 0.0017893979

“500 miles, or a little bit more.”

And you thought your delivery issues were obscure and hard to diagnose?

Trey has a FAQ about the story with some more details.

Ken has an article today about how Pro Flowers managed to get their mail out of the bulk folder at Gmail by asking their recipients for help.

This year, ProFlowers apparently took into account Gmail’s use of sender reputation and user engagement in its spam filtering rules by using subject lines, such as: “Gmail Customer Notice: Open if you missed yesterday’s special discount!” and “Help Teach Gmail to Like ProFlowers. Give us a Star.”

The data presented by Ken, with the help of eDataSource, shows that this strategy helped ProFlowers go from a less than 25% inbox rate on February 14th to a near perfect inbox rate by March 15th. This is a much faster rebuild of their reputation this year than in previous years.

Engaging with the recipient, and asking them for help, can and does affect reputation. In this case, Pro Flowers rebuilt their reputation and inbox delivery by asking the recipients to engage with them. ISPs want to deliver mail that their users want and interact with. Encouraging recipients to interact with mail, even in a non sender trackable fashion like “starring” in gmail, does improve delivery.

I checked our SenderScore earlier this month, as quite a few people mentioned that they’d seen SenderScore changes – likely due to changed algorithms  and new data sources.

It sure looks like something changed. Our SenderScore was, for a while, zero out of a hundred. That’s as bad as it’s possible to get. I didn’t get a screenshot of the zero score, but I grabbed this a couple of days later:

Are ReturnPath wrong? No. Given what I know about the traffic from our server (very low traffic, particularly to major consumer domains, and a negligible amount of unavoidable backscatter due to our forwarding role addresses for a non-profit to final recipients on AOL) that’s not an unreasonable rating. And I’m fairly sure that as they get their new algorithms dialed in, and get more history, it’ll get closer. (Though I’m a bit surprised that less than 60 mails a day is considered a moderate volume.)

But all our mail is delivered fine. I’ve seen none of my mail bounce. It’s very rare someone mentions that our mail has ended up in a bulk folder. I’ve received the replies I’ve expected from all the mail I’ve sent. Recipient ISPs don’t seem to see any problems with our mail stream.

A low reputation number doesn’t mean you actually have a problem, it’s just one data point. And a metric that’s geared to model one particular sort of sender (very high-volume senders, for example) isn’t going to be quite as useful in modeling very different senders. You need to understand where a particular measure is coming from, and use it in combination with all the other information you have rather than focusing solely on one particular number.

The Return Path In the Know blog listed 4 reasons mailing those old addresses is a bad idea.

Ashley, the author, is completely right and I endorse everything she said. (Although I’d really like to hear what happened to the customer that added back all those addresses. What was the effect on that campaign and future email marketing?) As I was reading the article though, I realized how many times this has been said and how depressing it is that we have to say it again. And again. And again.

A number of folks have told me that the reason they don’t pay any attention to delivery professionals is because we don’t provide enough real data. They can show that sending mail to old addresses costs them nothing, and makes them real money.

That’s not really true, though. We do provide data, they just don’t like it so they don’t listen to it. Return Path publishes lots of numbers showing that mailing unengaged recipients lowers overall delivery. I can provide case studies and data but companies that are committed to sending as much mail as possible throw up many reasons why our data isn’t good or valid.

The biggest argument is that they want hard numbers. I do understand this. Numbers are great. Direct and clear answers are wonderful. But delivery is a squishy science. There are a lot of inputs and a lot of modifiers and sometimes we can’t get exactly one answer. The data is noisy, and difficult to replicate. One of the reasons is that filtering is a moving target. Filters are not, and cannot be, fixed. They are adaptive and are changing even between one hour and the next.

Delivery experts are about risk management. They are the parents requiring everyone in the car wear seat belts, even though the driver has never had an accident. They are the fire department enforcing fire codes, even though it’s the rainy season.

Risk management isn’t about the idea that bad things will absolutely happen but rather that it is more likely that a bad thing will happen in some cases.

In this case, it’s more likely that delivery problems will happen when mailing old addresses. And if those addresses aren’t actively contributing to revenue, it’s hard to argue that their presence on a list is more beneficial than their absence.

But I repeat myself. Again.

In many cases these days, many more cases than a lot of senders want to admit, delivery problems at the big ISPs are a result of sending mail recipients just don’t care about. The reason your mail is going to bulk? It’s not because you have minor problems in your headers. It’s not because you have some formatting issues. The reason is because your recipients just don’t care if the ISP delivers your mail or not.

A few years ago the bulk of my clients hired me to do technical audits for their mail. I fixed a lot of delivery problems that way. They’d send me their email and I’d run it through tools here and identify things they were doing that were likely to be causing problems. I’d give them some suggestions of things to change. Believe it or not, minor tweaks to headers and configuration actually did make a lot of difference in delivery.

Over time, though those tweaks less effective to fix delivery problems. Some of it is due to the MTA vendors, they’re a lot better at sending technically correct mail than they were before. There are also a lot more people giving good advice on the underlying structure and format of emails so senders can send technically clean email. I started seeing technically perfect emails from clients who were seeing major delivery problems.

There are a number of reasons that technical fixes don’t work like they used to. The short version, though, is that ISPs have dealt with much of the really blatant spam and they can focus more time and energy on the “grey mail”.

This makes my job a little harder. I can no longer just look at an email, maybe run it through some of our tools and provide a few suggestions that fix delivery problems. Delivery isn’t that simple any longer. Filters are really more focused on how the recipients react to mail. That means I need to know a lot more about a clients email program before I can even start to identify what might be causing the delivery issues.

I wish it were still so simple I could give minor technical tweaks that would appear to magically improve a client’s delivery. It was a lot simpler process then. But filters have evolved, and senders must evolve, too.

Return Path released their 3rd quarter email intelligence report this week. And the numbers aren’t looking that great for marketers.

Complaints are a major problem for commercial mailers. In the data Return Path examined, commercial mail made up 18% of the total inbox volume. That same mail accounted for 70% of all email complaints.

Additionally, 60% of the email sent to spamtraps was commercial email.

The combination of complaints and spamtrap hits mean 16% of commercial email doesn’t make it to the inbox.

And, as no surprise to anyone, Postini is the worst performing B2B filter out there. Folks behind postini filters only get 23% of the email they’re sent in their inbox. And 44% of that mail is just outright lost.

Ken Magill article.

DMNews article.

Over on the AOL Postmaster blog, Lili Crowley announced yesterday that AOL has made changes to their spam filtering system. Specifically, more senders may be subject to blocking with CON:B1 errors. AOL’s website explains that CON:B1 errors indicate that an IP address is being  blocked “due to a spike in unfavorable e-mail statistics.” This strongly suggests that a sender blocked with a CON:B1 error message has a negative sending reputation. This is yet another data point as to how ISPs have been tightening up spam filtering and reputation requirements over the past few years. What you might have been able to deliver five years ago, you might not be able to get delivered today.

Bronto’s Chris Kolbenschlag frames the discussion well: He purchased from an online retailer, they assumed he wanted to receive followup emails, and thus, those emails did eventually commence.

This is something I’ve had a lot of experience with. Working for an e-commerce service provider from later 2000 through mid 2006, I was the guy setting permission policy, dealing with spam complaints and advising on deliverability issues, primarily regarding email lists built over time from online store purchasers. There was an opt-in checkbox on the platform’s checkout pages, and it was up to the client as to whether or not it was pre-checked (“opted-in”) by default. Most clients pre-checked it by default.

My experience was, from a deliverability perspective, this kind of auto opt-in didn’t really present issues. People didn’t tend to forge addresses when purchasing, and people tended not to report mail as spam when it’s coming from somebody they just did business with.

I’m not saying it’s the wisest way to do things, by any means. If you have any other deliverability challenges at all, this kind of thing could likely add to them. And is it the most consumer friendly way to run things? I don’t think so. In my humble opinion, it’s always better to wait for the consumer to sign up on their own. But I’m not one of those aggressive marketer types.

And of course, the laws governing email permission vary by locale.

I am one of the moderators of a discussion list working on a document about getting off blocklists. If anyone not on the list attempts to post to the list I get a moderation request. One came through while I was gone.

Now, I don’t really think Jim Mills wants to be friends with a mailing list. I think he probably gave LinkedIn his email password and LinkedIn went through and scraped addresses out of his address book and sent invitations to all those addresses.

I don’t have any problem with connecting to people on social media. I do even understand that some people have no problem giving their passwords over to let social media sites plunder their address books and find connections. What I do have a problem with is social media sites that don’t do any pruning or editing of the scraped addresses before sending invitations.

In this case, the email address, like many mailing lists, has in the email address “mailman.” While it’s probably impossible to weed out every mailing list, support address and commercial sender, it doesn’t seem like it would be too difficult to run some minor word matching and filtering. It’s not even like those addresses have to be removed from invites. Instead they could be presented to the user for confirmation that these are real people and addresses.

Yes, it’s friction in the transaction and it costs money to do and do well. But those costs and friction are currently offloaded onto uninvolved third parties.

I’ve been thinking a lot about the field of ISP relations and what it means and what it actually is. A few years ago the answer was pretty simple. ISP relations is about knowing the right people at ISPs in order to get blocks lifted.

The fact that ISPs had staff just to deal with senders was actually a side effect of their anti-spam efforts. In many places blocking was at least partially manual, so there had to be smart, technical, talented people to handle both the blocking and unblocking. That meant there were people to handle sender requests for unblocking.

Spam filters have gotten better and more sophisticated. Thus, the ISPs don’t need smart, technical, talented and expensive people in the loop. Most ISPs have greatly scaled back their postmaster desks and rely on software to handle much of the blocking.

Another issue is that some people on the sender side rely too heavily on the ISPs for their data. This makes the ISP reps, and even some spam filtering company reps, reluctant to provide to much help to senders. I’ve had at least 3 cases in the last 6 months where a sender contacted me to tell me they had spoken with someone at an ISP or filtering company and were told they would get no more help on a particular issue. In talking with those reps it was usually because they were drowning under sender requests and had to put some limits on senders.

All of this means ISP Relations is totally different today than it was 5 years ago. It’s no longer about knowing the exact right person to contact. Rather it’s about being able to identify problems without ISP help. Instead of being able to ask someone for information, ISP Relations specialists need to know how to find data from different sources and use that data to identify blocking problems. Sure, knowing the right person does help in some cases when there’s an obscure and unusual issue. But mostly it’s about putting together any available evidence and then creating a solution.

We still call it “ISP Relations” but at a lot of ISPs there is no one to contact these days. I think the term is a little misleading, but it seems to be what we’re stuck with.

Campaign Monitor analyzed over 2.2 million campaigns and came up with some rules of thumb for effective email marketing.

Volume in an of itself is not a problem. Companies sending mail people want can send multiple emails a day to every user. The volume isn’t a problem because the mail is wanted.

Many senders are confused and think volume is a filtering criteria. It’s not. Send all you want; just send it to people who actually want the mail.

A lot of companies in their growth phase find they do have delivery problems as their volume ramps up. But the problem isn’t the volume, the problem is that mail programs don’t scale. Companies mailing lower volumes can get away with sloppier practices. One because the chances of hitting bad addresses increases with the number of addresses you have. But the other is that filters do take volume into account. It’s not that the volume directly causes the filters to trigger, but volume causes the filters to look harder at mail. If the reputation and metrics are good, the mail is fine and hits the inbox. If they are poor, then mail hits the bulk folder or is filtered.

Overall, volume isn’t a problem, but increasing volume can expose fundamental problems in a mail program that result in delivery issues.

The post Is volume a problem? appeared first on Word to the Wise.

Campaign Monitor has an interview with Gmail looking at how to get mail to the Gmail inbox. It’s a great article and I think everyone should go read it.

One of the most important things it talks about is how complex filters are.

On Gmail’s end, Sri revealed that there are literally hundreds of signals to decide whether an email should go to the Inbox or the Spam folder. The importance of any given signal is dynamic and determined on complex algorithms, in essence it means that one factor or another isn’t likely to bin an entire campaign and there is no point in obsessing over any one element. “Think of how you can make the user love your mails rather than how to land in the Inbox” was Sri’s basic advice on the subject. Essentially stating if the user likes your mail the spam filter should not stop it from getting to the Inbox.

This really is the crux of delivery. Send email users want to receive. Sri’s statements to Andrew echo many of the things he, and his team, shared with us at M³AAWG in February. I focused more on the technical things but engagement and mail users want to receive was an ongoing theme through the talk.

Gmail is often the toughest inbox to crack because they rely so heavily on engagement metrics. But engagement as a metric for delivery is nothing new. I’ve been writing about how engagement is critical for delivery since at least 2008. I have posts from 2011 talking about how to increase engagement and inbox delivery.

I know that engagement and relevance are bad words in the marketing space. An number of marketers have made very public statements about how relevance is dead and engagement is something bad consultants have made up to keep them in business. The fact of the matter is that engagement is something the ISPs do look at and do measure. Anyone who wants to have a successful email marketing program needs to look at what their users want to receive. Sending mail users want leads to inbox delivery because that’s what makes the ISPs money.

The post More from Gmail appeared first on Word to the Wise.

Mention an email is spam to some senders and watch them dance around trying to explain all the ways they aren’t spammers. At some point, calling an email spam seems to have gone from a statement of fact into some sort of moral judgement on the sender. But calling an email spam is not a moral judgement. It’s just a statement of what a particular recipient thinks of an email.

There are lots of reasons mail can be blocked and not all those reasons are spam related. Sometimes it’s a policy based rejection. Mailbox providers publishing a DMARC record with a reject policy caused a lot of mail to bounce, but none of that was because that user (or that mailing list) was sending spam. Most cable companies prohibit customers from running mail servers on their cable connection and mail from those companies is widely rejected, but that doesn’t mean the mail is spam.

Sometimes a block is because some of the mail is being sent to people who didn’t ask for it or are complaining about it. This doesn’t make the sender a bad person. It doesn’t make the sending company bad. It just means that there is some issue with a part of the marketing program that need to be addressed.

The biggest problem I see is some senders get so invested in convincing receivers, delivery experts and filtering companies that they’re not spammers, that they miss actually fixing the problem. They are so worried that someone might think they’re spammers, they don’t actually listen to what’s being said by the blocking organization, or by their ISP or by their ESP.

Calling email spam isn’t a moral judgement. But, if too many people call a particular email spam, it’s going to be challenging to get that mail to the inbox. Instead of arguing with those people, and the filters that listen to them, a better use of time and energy is fixing the reasons people aren’t liking your email.

The post Spam is not a moral judgement appeared first on Word to the Wise.

Occasionally when we meet longtime readers of the blog at conferences and industry events, they are surprised to learn that we are not just bloggers. We actually spend most of our time consulting with companies and service providers to optimize their email delivery. Though we try to avoid using the blog as a WttW sales pitch, we thought it might be useful to devote a short post to explaining a bit more about what we do.

Most of the companies we work with have strong email marketing and technical expertise, but face challenges beyond normal “best practice” recommendations. To get started, companies often engage with Word to the Wise in one of two ways:

• Technical Audit: a short-term engagement designed to identify and resolve any underlying issues that may be causing delivery problems. We will look at message and header structures, content issues, and sender reputation or authentication problems.

• Strategic Consulting: Some of our clients have dubbed this “email therapy”, and these engagements are structured a lot like that – we speak once or twice each week for several months about our client’s specific challenges and collaborate closely to work on larger programmatic and technical issues around optimizing email programs. We really enjoy this type of deep forensic work and helping clients create more sustainable programs.

That said, we very much believe that there’s no one-size-fits-all approach to this sort of work. We are also happy to work with companies that don’t have established expertise in-house, or who are in the process of developing email programs. We have a wide range of additional services we provide, from data analysis to ISP/ESP relations to blocklist resolution.

In general, we want to provide support for email program managers and systems administrators to create and manage meaningful and valuable customer communication. Our experience working with a broad range of senders over many years, as well as our close relationships with ISPs, ESPs, spam fighters and blocklists, policy and governance bodies, and other email consultancies gives us both broad and deep insight into the current landscape of email. If you think your company could benefit from this type of support, please get in touch!

The post What We Do appeared first on Word to the Wise.

Yesterday I talked about how data collection, management, and maintenance play a crucial role in deliverability.  I mentioned, briefly, the idea that bad data can accumulate on a list that isn’t well managed. Today I’d like to dig into that a little more and talk about the non-permanence of email addresses.

A common statistic used to describe list churn is that 30% of addresses become invalid in a year.  This was research done by Return Path back in the early 2000’s. The actual research report is hard to find, but I found a couple articles and press releases discussing the info.

A new study, to be unveiled next week at The 85th Annual Direct Marketing Association conference, indicates that email addresses are changing at the rate of 31% annually, driven by ISP switching, job changes and consumer efforts to avoid SPAM.

The email survey, conducted by independent, third-party research firm NFO WorldGroup, concluded that, consequently, the majority of consumers lose touch with personal and professional contacts and with preferred websites. […]

The survey, conducted in August 2002, updates a similar study by Return Path and NFO WorldGroup from September 2000, which identified a 32% annual rate of email address churn. The results are based on responses from 1,015 consumers from NFO WorldGroup’s online panel of U.S. email users over the age of 18. The panel is representative of U.S. online households.  ISP Switching and SPAM Continue to Drive Email Address Changes

While I think the address change rates are probably lower now, list churn still exists.

In 2002, NFO reported users changed personal email addresses for a number of reasons.

• 50% changed due to an ISP switch
• 16% changed due to spam
• 12% changed due to a move
• 8% changed due to a “more attractive” email address.

Work users also changed addresses, and for many of the same reasons.

• 41% changed due to new jobs
• 18% changed due to an ISP change
• 8% changed due to a residential move
• 6% change due to a name change (divorce or marriage)

Given the changes in free webmail providers since 2002, I expect address changes due to ISP switching or moving is less common than it was. But other reasons that users cited still exist, including spam levels, new jobs and name changes.

Of course, my gut feeling that these numbers are old and out of date and probably no longer accurate was crushed last week. The LA Times published an article about Hillary Rodham Clinton’s email campaigns. After her run for president in 2007, her email address database had approximately 2.5 million records. According to the article, less than 100,000 of the addresses are still valid. That’s more than 30% attrition every year.

List churn is real. While we may not know what the exact percentage of churn is, we know it happens. I expect that list churn, like most things in deliverability, is related to the actual recipient group. Some lists, like Secretary Clinton’s list, may have a very high churn rate. Other lists focused on different demographics might have a much lower churn rate.

While the LA Times article mentioned these addresses bounced (“an inbox clogged with bounce-back messages”) not all churn is so visible. There’s also “stealth” churn, where addresses are abandoned by their users but still accept mail.

What can you do? Mostly I recommend first wrapping your head around the idea that churn exists. Once you really believe churn is real then you can address how to fix it in your specific environment.

Key things to remember when planning a data management plan:

• Email addresses are not permanent.
• Subscriber data degrades if you don’t actively manage it.
• Deliverability depends on data quality.
• Maintaining data is easier than trying to clean data.
• Using list cleaning services will remove hard bounces, but won’t address “stealth” churn, which can still affect deliverability.

If there’s anything my work with clients has taught me is that the more creative and flexible you can be in regards to list management, the more effective your overall email marketing program can be.

The post Yes, Virginia, there is list churn appeared first on Word to the Wise.