Quantcast
Channel: Delivery Improvement – Word to the Wise

Filters do what we tell them

0
0

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.

What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.

Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”

Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.

Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.

Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.

Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.

Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

The post Filters do what we tell them appeared first on Word to the Wise.


Permission trumps good metrics

0
0

Most companies and senders will tell you they follow all the best practices. My experience says they follow the easy best practices. They’ll comply with technical best practices, they’ll tick all the boxes for content and formatting, they’ll make a nod to permission. Then they’re surprised that their mail delivery isn’t great.

Too many senders, ESPs and deliverability services companies, believe that the key to the inbox is checking all the best practice boxes. List hygiene and list cleaning companies are the most obvious example. Bounces are a key reason for bad delivery. If we remove all or most bounces then our numbers comply with the standard metrics. If our complaints are low then we comply with the standard metrics. If our metrics all look right, then we’re clearly doing everything right and we should reach the inbox.

That’s really not the case, though. Good delivery is much more than just hitting the right metrics. Good delivery is more than doing the technical stuff right. Good delivery requires sending mail people want, and much of that can’t be measured in bounces or complaints.

As an example, there’s an ESP I only discovered because I received mail from their customers. Don’t know anyone working there, have never heard of them before. All I know about them is in my inbox.

Technical check

I’ll give the ESP this, they have their customers technically set up correctly. Going through the tests I do when auditing clients I can’t find anything really wrong.

  • Each customer gets their own d= domain.
  • That d=domain aligns with the from address.
  • SPF is set up correctly.
  • SPF validates
  • MessageID is correctly formatted
  • HTML looks reasonably clean
  • ListUnsub header is present
  • mailto: and href: links in proper order
  • CAN SPAM address
  • Unsub link works

There’s even a website on the domain used for SPF authentication. It’s not horribly useful, but it’s there.

It says:

Congrats on your hacking skills!

Wonder what this site is? We are an award winning high quality ESP – an Email Marketing Service.

We help companies maintain their brand with Marketing as well [sic] Transactional messages.

EMAIL.DELIVERED.PERIOD.

(Note: I don’t believe taking a domain name and typing it into a browser bar is hacking. I don’t think being able to read full headers is hacking.)

This email meets all the technical standards. If I had to guess, I’d say that the bounce rates are low. I expect complaint rates are also very low. Overall, these senders are following all the standard best practices and if I had to score them just on meeting technical standards I’d give them a 10/10.

Deliverability check

That’s the technical piece of delivery. What else matters for delivery? Things like format, content, and relevance.

Format wise, the messages themselves are text, not plain text but nicely formatted business style text. They almost look like personal mail. Nothing remarkable, but probably a good fit for the busy small business person.

Content wise, it’s well written copy. Each of the senders clearly put some work into the wording and phrasing. It’s not something they just dashed off, but doesn’t look overly polished. Again, nothing remarkable but probably a good fit for the audience.

As I am the audience for three of these messages, I get to decide if these messages are relevant.

  • One is selling me a plugin for Outlook to “transform my sales process.” Well, I won’t use Windows for email and I don’t have Outlook installed on my mac. So that’s not very relevant to me.
  • Another is selling me qualified sales leads. Almost all of our business comes through word of mouth and recommendations from industry folks. So that’s not very relevant to me.
  • The third is from my BFF on LinkedIn. He writes articles on investing, music, and life and will subscribe me unless I tell him stop. Even though he claims he’s my BFF, his opinions aren’t very relevant.

Overall, it’s a strong showing in formatting and content with a definite lack of relevancy. Overall, I give it a 8/10.

Permission check

And here’s where we get to the problem. None of these senders have permission to email me, and they certainly don’t have permission to email me at that address. There’s not much to say here other than to give them a 0/10 on permission.

Compliance Check

ESPs have two big roles in deliverability: technical and compliance. I mentioned the technical above and they’re doing stuff right. There are a few things I can’t see from receiving emails, like throttles and connection limits, but I suspect they’re right in the mainstream there as well.

The compliance piece is actually a big part of what makes deliverability from specific ESPs good. The reason might surprise people. ESPs do have reputations, but they aren’t the same as sender reputations. An ESP builds their reputation by effectively dealing with problem customers. Everyone leaks, bad mail comes out of every network at one time or another. Spamhaus, filtering vendors and ISPs know this. But they also know that some ESPs monitor and police their customers more than others. These ISPs often get the benefit of the doubt before blocks go up (dot zero listings for instance)

The ESP they’re using does have a decent looking AUP, evening mentioning they use the Spamhaus definition of spam. Unfortunately, I reported two emails to abuse@ and received a disappointing response. All they said was they would suppress my email address.

This is disappointing. I mean, it’s great that they’re going to suppress my address. But that doesn’t address the broader issue: their customers are sending mail in violation of their AUP. Last week I mentioned a complaint to an ESP (again, one I’d never heard of) that sent me back a message that said, “Thank you for notifying us, we take these issues seriously.  I’m investigating with the sender and will let you know when its resolved.” And they did!

I’ll give their compliance a 3/10, because at least they’re suppressing my address.

Overall

Adding up the scores I get 21/40. OK, so this is a somewhat arbitrary scale. But, the point remains, permission is critical to delivery. You can do all the technical stuff and content stuff right, but if you fail to get permission delivery is going to suffer. And if your ESP isn’t up on compliance, then they’re not doing you any favors.

B2B spam is still spam. Spam isn’t defined by what’s in it or by whether it’s authenticated or if it’s has the right metrics. Spam is unsolicited email. Permission is key. Permission trumps all.

The post Permission trumps good metrics appeared first on Word to the Wise.

Filtering by gestalt

0
0

One of those $5.00 words I learned in the lab was gestalt. We were studying fetal alcohol syndrome (FAS) and, at the time, there were no consistent measurements or numbers that would drive a diagnosis of FAS. Diagnosis was by gestalt – that is by the patient looking like someone who had FAS.

It’s a funny word to say, it’s a funny word to hear. But it’s a useful term to describe the future of spam filtering. And I think we need to get used to thinking about filtering acting on more than just the individual parts of an email.

Filtering is not just IP reputation or domain reputation. It’s about the whole message. It’s mail from this IP with this authentication containing these URLs.  Earlier this year, I wrote an article about Gmail filtering. The quote demonstrates the sum of the parts, but I didn’t really call it out at the time.

Gmail uses a 10+ year old neural network that analyzes thousands of factors, related to email, IP, and web, integrated with all Google products, and with 99.9%+ accuracy for identifying certain types of messages, combined with an email-specific domain-based reputation system that combines IP reputation, content, read rates, reputation of other senders with similar content.

With filters, Gmail looks at the whole picture. They look at all the data and assess the whole.  Gmail filters by Gestalt. I think other companies are catching up and this is the filtering of the future.

So… what’s that mean?

That means that we’re not looking at warming up an IP or a domain. Instead we’re warming up a domain on an IP. Take the domain to another IP, and the reputation doesn’t carry. Change a domain on an IP and that needs to be warmed up as a domain/IP pair.

But even that is overly simplified from reality. It’s not a domain/IP pair, it’s this SPF domain, that d= domain, this IP, this DMARC alignment, these URLs, and on and on. A recent talk referred to warming up resources in relationship to each other, where resources were things like IPs, domains, and URLs.

Spamassassin with relative scores

I think most readers have a good feeling for how Spamassassin works. It has a bunch of rules, and assigns scores based to each rule. All the scores are added together and if they’re higher than a certain value the mail is filtered.

In more modern filtering, particularly at Gmail, scoring is dynamic. There are still rules and they still assign scores. But the scores themselves can be modified by other scores in the process. It’s not a simple sum of scores so changing anything can change the overall status of a message.

Take two identical messages and two IP addresses one with an arbitrary reputation of 5 and another with an arbitrary reputation of 10. By the score and sum method, the final email reputation scores would be message+5 and message+10. With relative scoring, though, the IP reputations might turn out to be 2 and 13.

Look at the whole picture

There’s a West Wing episode where Jeb is playing chess with multiple members of the White House staff while negotiating the international crisis of the week. Throughout the episode he tells staff to “look at the whole board.” This is really what we have to be doing in deliverability right now. We have to look at the whole board. We have to look at the whole face. We have to see the gestalt.

We can’t just look at the domains and URLs in a message, we have to consider them in context with the IP addresses. All mailstreams affect each other. No longer can we look at transactional messages as separate from marketing messages. The reputation of each affects the other.

This is actually good. It means that different mailstreams, even with the same URLs from the same IPs can develop independent reputations. It makes it easier to use shared IPs. Reputation isn’t reliant on keeping everything separate. It’s the whole picture that’s important.

Email is much more than the sum of its parts.

 

 

The post Filtering by gestalt appeared first on Word to the Wise.

5 steps for addressing deliverability issues

0
0

excFollowing on from my reading between the lines post I want to talk a little bit about using the channels. From my perspective the right way to deal with 99% of issues is through the front door.

Last week I found myself talking to multiple folks in multiple fora (emailgeeks slack channel, mailop, IRC) about how to resolve blocking issues or questions. All too often, folks come into these spaces and start by asking “does anyone know someone at…” Fundamentally, that’s the wrong first question. Even if the answer is yes. It’s even the wrong question if a representative of the company is on the list where you’re asking for help.

If that’s the wrong question, what is the right question? Where can we start to get help with issues when we’re stuck trying to fix a delivery problem we don’t understand?

1: Read.

Read the full bounce message. The first step is always reading the full bounce message. ISPs are pretty good at providing information in their bounce messages. Look at the full message, and follow any links. Read the information. The links are typically designed for the folks who work in the industry. This means sometimes the language might be jargon. It can take a little work to understand but the help is intended to be there.

In many cases, these information pages will contain links to contact forms for further questions. They’re often not accentuated like a typical call to action. This is intentional. If the visitor is skimming and looking for a contact us button, they’re not actually reading the information on the page. Companies put a lot of time into creating these pages, and try to cover most of the common issues and resolutions in them. Most of the time these pages cover the issue a particular visitor is having.

2: Use the form.

In those cases where the information on the page doesn’t seem to apply, the next step is to use the contact form. Sometimes these forms seem wildly inappropriate, and ask for all sorts of strange bits of information unconnected from the problem. Still, it is best to fill out the form as completely as possible.

There are certain bits of information that are vital for troubleshooting an issue. Things like the sending IP, the domain authentication, any special codes in the bounce string help the sender address the issue. Without those bits of information, it’s nearly impossible for the ISP to answer questions and resolve the blocks.

2a. I can’t find the form.

If you can’t find the form there are a couple things to check. One is to do a text search (⌘-F or alt-F) and search for “contact” or “form” to find the actual link. I do this, sometimes, when I’m in a hurry and my eyes are glazing over the text and I keep missing the link. The second is to search the web. I maintain a list of postmaster pages and links (which is less maintained than I’d like, but I’m working on that). I’m also not the only person who aggregates that data, although most of the links I can find right now focus on the FBL signup pages (ASRG, MAAWG, and Wikipedia).

Sometimes there isn’t a form to fill out. Often this is because the maintainer doesn’t want to or won’t answer questions. There isn’t much to do in these cases.

3: Ask around.

The the previous steps haven’t worked, reaching out for help is the next step. It’s very common for a lot of technical folks to hang out in online spaces to answer questions to help those learning. In the email space, I’d say that was mailop. I regularly see questions in a lot of different places, like public, private and semi-private mailing lists, and slack channels.

There are right ways and wrong ways to ask for help in these fora. That’s probably a whole blog post in itself, but let’s look at some of the highlights.

  • Provide the full bounce message
  • Provide as much information about your network as possible, including domains and IP addresses.
  • State what you’ve done
  • State how long (roughly) the problems’ been going on.

Notice I didn’t add in a state what kind of help you want or will accept in that list of bullet points. All too often messages come in looking for direct personal contacts. That’s usually not going to happen, particularly if no one knows you. I’ve blogged about why using personal contacts is bad practice before: Use the form, Follow the script. As J.D. points out in the comments of the second blog post, some of the technical folks shouldn’t be customer facing, so using the channels is better for everyone.

4. Listen.

The answers we get back from requests are not always the answers we want. I mentioned a few of the issues in last week’s blog post. They’re not the only problems. The biggest problem I see is senders not wanting to believe what’s there in black and white. They don’t want to hear the answer or believe it.

I get it. It’s hard to believe that people don’t want that carefully crafted and targeted email. Therefore, the filters must be wrong, it must be a mistake. More often than not, though, the filters are catching the mail they’re designed to catch.

There are, of course, cases where the filters are wrong. Generally that is the only place to use trusted back channels. The folks managing the filters don’t want to hear or listen that they’ve screwed up any more than email senders want to hear it. But when someone who never argues about a filter or a listing sends and email  asking if they meant to block a particular class of mail, those inquiries are taken seriously. Sometimes, as with the listboming SBL listings, the answer is yeah, actually, we did mean to do that. Other times it’s a ooh, no, let’s fix that.

5. Interpret and act.

The final step is to take the information and act on it. This can be a challenge as often the replies don’t list a set of changes to make. They’re never going to be specific. To quote a post from the mailop list:

What we do for one, we must do for all.
If we can't do it for all, we can't do it at all.

While this statement is from a single ISP, I believe the sentiment is broadly applicable. ISPs do not and will not provide step by step instructions for delisting. They can’t. The minute ISPs start sharing that type of information spammers will take advantage of that. Once again, we all suffer because spammers are jerks.

The good news is dozens of websites, including this one, provide free advice and assistance on how to fix delivery problems. ESPs have extensive internal documentation for customers. Many ESPs have experts on staff to help customers.

Even better that all the free and included resources, there is usually one underlying issue causing delivery problems. Conceptually it’s easy to fix deliverability problems. Delivery fails because recipients aren’t excited about the messages. Solving delivery problems boils down to sending mail recipients expect and want to receive. Figure out how to do that and you’ve solved the long term problem. Solving the short term problem means focusing mailing engaged users.

 

 

The post 5 steps for addressing deliverability issues appeared first on Word to the Wise.

Improving Gmail Delivery

0
0

Lately I’m hearing a lot of people talk about delivery problems at Gmail. I’ve written quite a bit about Gmail (Another way Gmail is different, Gmail filtering in a nutshell, Poor delivery at Gmail but no where elseInsight into Gmail filtering) over the last year and a half or so. But those articles all focus on different parts of Gmail delivery and it’s probably time for a summary type post.

Gmail is different

There are two major reasons that Gmail filtering is different from the other webmail providers: when it was launched and who it was launched by.

Gmail entered the mail market late in the internet era when compared to other free email providers. AOL offered internet email in 1992; Yahoo Mail opened in 1994; Hotmail debuted in 1996. When these systems were in development, spam wasn’t an issue.
Spam filtering was added later, as the problem grew. Gmail didn’t launch until 2004, nearly a decade after their current competitors. Spam was already a problem by 2004, so Gmail was able to build filters in from the beginning.

The other real difference is Google’s experience and expertise in search. They built their business on being able to take lots and lots of data, categorize it and make it instantly searchable. This actually translates well to spam filtering, in that they take lots of data, categorize it and put it in appropriate mailboxes.

Those aren’t the only reasons Gmail is different. Another factor is Gmail’s attitude towards senders. The prime example is their FBL. Unlike most ISPs, Gmail doesn’t provide the full message back in its FBL. Instead, they give a count of complaints. They’re not going to help senders remove folks who complain. The flip side of this is they are leading the way in providing easier ways to unsubscribe.

The different history, expertise, and attitude of Google are the core of why Gmail delivery is so unlike others.

Metrics look great

The standard diagnostic for problem is to investigate the metrics, identify areas where they show limits and work to improve them. Along the way, email delivery improves. At Gmail, however, there’s often nothing obviously wrong with the metrics. The problem is the metrics we’re using are measuring symptoms not identifying underlying issues. Think of all the metrics we use as a fever. Just because a fever is gone (or you don’t have one) doesn’t mean you’re not sick.

Metrics are proxy measurements. The best metrics in the world aren’t going to help your delivery at Gmail if the recipients don’t want your mail.

The Recipient Has To Want Your Mail.

Why is Gmail so hard?

Because Gmail is smarter than we are.

Because Gmail looked at the things other companies did and learned what worked to decrease spam and what worked to decrease signs of spam (those are different things).

Because Gmail has years and years of experience in dealing with people who game SEO listings.

Because Gmail puts the user experience ahead of the sender experience.

The post Improving Gmail Delivery appeared first on Word to the Wise.

Interacting in professional fora

0
0

There are a bunch of online communities – mailing lists, Slack channels, etc. – where “people who do email” interact.

Some of them are open to anyone to subscribe, some of them are semi-private and require an invitation, others are closed and only available by invitation and yet others are associated with trade associations and only open to their members.

Many of them include representatives from ISPs, ESPs, reputation providers and technical specialists. They also – especially the open lists – have participants with no particular role in the industry, but very strong opinions on what others should do.

They’re a useful place to keep up to date on current issues and industry trends, and to get help when you need it. But … quite a lot of people reduce their chance of getting timely help by the way they behave there. Don’t be like those people.

Some of the things you should and shouldn’t do are specific to mailing lists. Some are specific to professional fora. Some are specific to entreating others for help. Here, in no particular order, are some suggestions:

 

DO: Be friendly. Be patient. Be welcoming. Be considerate. Be respectful.

DO: Be careful in the words that you choose.

DON’T: Be a dick.

DON’T: Be wildly unprofessional. If you think sexist or racist behaviour isn’t wildly unprofessional, leave the email industry. Ditto for unwanted sexual attention, personal insults, sexualized language or imagery.

DON’T: Harass people. If someone wants you to stop, then stop.

 

 

DO: Follow the community norms. Different communities have different styles and traditions – try and pick up on what they are, and avoid violating them.

DO: Follow the community norms for replying to messages, quoting them and trimming threads. If you’re not sure what they are then snipping out parts that aren’t relevant and replying in-line isn’t likely to offend anyone.

DO: Follow the level of formality of the community. Some are very formal, and should be treated much the same as a business meeting. Others much less so, and blend professional discussion with blowing off steam, ranting about idiot clients and social banter between friends.

DO: Lurk on the list for a day or three before posting to get a feel for how the community works (unless there’s a “welcome to the new person” thread). If you’ve joined because you have an immediate emergency you’re looking for help on, say so and be polite – maybe even a little apologetic – about it. Maybe spend five minutes checking the list archives first.

DON’T: Lurk except when you have a problem. Interacting with others when you’re not asking for help builds up relationships and karma. If you only appear when you’re looking for help, people are less likely to be helpful.

 

 

DO: Be clear about what company or organization, you’re affiliated with. That might mean using a corporate email address, mentioning it in a sig file or in a “Hi, I’ve just joined the group” message. Or it might mean including the relevant company name when asking for help. If, for political reasons, you absolutely cannot admit to your affiliations it’s still useful to know that you work for an unnamed major US cable company or an email provider based in Switzerland – particularly when you’re offering help or advice where your insight is coming from your experience in that role.

DO: Remember that the vast majority of the people you’re interacting with aren’t being paid to be there. They’re sharing their time and expertise in return for benefiting from others. Try to both give and take.

DO: Remember that a representative from a large ISP probably doesn’t have answering your questions or helping with your problem in their job description.

 

 

DON’T: Aggressively demand help. Nobody owes you anything.

DO: Read responses carefully. Someone may not be able to publicly join the dots on an issue for you, but may point out which dots you might want to look at.

DO: Understand limits. If someone says “our lawyers say this is the process you must follow” then follow that process. And don’t push that person to do things that their lawyers say they can’t do.

DO: Be aware that you’re interacting with people, not company representatives. They almost certainly have opinions that don’t reflect those of their organizations.

DO: Remember that nobody owes you support. Be nice. And if someone doesn’t volunteer help or stops responding, don’t badger them.

 

 

DO: Follow the community style for how you present your message. But … in general, mostly plain text won’t offend anyone, heavy use of rich text will annoy some people.

DON’T: Rely on rich text for meaning. It may not be visible to some people or not visible when quoted. “Look at the log lines highlighted in yellow” isn’t a good approach.

DON’T: Warlord. There’s no need for long legal disclaimers on your mail. Nor for more than four lines of signature – we don’t need to know your life history. Graphics are cheesy, even if they’re your employers professionally drawn logo. Even colour can be distracting if it’s not used carefully.

 

DON’T: Assume that you’re the best representative of your organization to interact with a community. If you’re a senior manager and you have a smart employee who is actively working in the area – they may be a better rep than you are.

 

DO: Be aware of how public a community is. Does it have a public archive that’s indexed by Google? Is it open subscription? Be aware of how public things you say are.

DO: Be aware of what is expected from you in terms of information distribution. Can things you learn from the community be shared elsewhere? With attribution, or not? If you’re not sure, don’t share information unless the person providing it OKs that – it’s always OK to ask if you’re not sure. Terms you might see are Traffic Light Protocol or Chatham House Rule.

 

DO: Assume good faith.

 

DO: Provide relevant information when looking for help or asking “has anyone else seen this?”.

DO: Check unread mail to a list before posting. If someone else is already talking about an issue, join that thread rather than starting your own.

DO: Check the archives first, if you can. The answer to your problem might be in there. And if it’s not, including a mention of “this looks similar to what Yahoo was doing in October” signals that you’ve done a little work before asking for help and might trigger someone’s memory of what happened last time.

DO: Include relevant IP addresses and hostnames, if you’re asking about a delivery issue.

DO: Include exact error or rejection messages – “blocked at AOL” isn’t particularly useful, “554 RLY:B1” is much more so.

DO: Mention what sort of email it is, especially if you think the problems may be content related.

DON’T: Obfuscate.

DO: If you’re asking about a problem, say how long it’s been going on and what you’ve already tried to fix it.

DO: Respond promptly if someone asks for more details.

DON’T: Expect help if you’re not prepared to share data.

DON’T: Vanish once you resolve the problem. Share what you did, even if it’s just “it cleared up around 3pm”.

All long help threads should have a sticky globally-editable post at the top saying 'DEAR PEOPLE FROM THE FUTURE: Here's what we've figured out so far ...

DO: Be prepared to take conversations that only you and one other person, out of hundreds, are interested in to direct message or private email.

 

DO: Stick around and help others. Share what you know.

DON’T: Post off-topic stuff people aren’t going to be interested in. It’s great that your kid is selling girl scout cookies or you’re doing a charity 5k, but unless you’re absolutely sure that this is a good place to fundraise, it almost certainly isn’t.

DO: Keep conversation on a mailing list, on the mailing list. There’s no need to Cc everyone involved – they’re on the mailing list too.

 

DON’T: Email angry. If someone has made you mad, wait before responding.

The post Interacting in professional fora appeared first on Word to the Wise.

Tempo

0
0

When we say that you might just be sending too much email and fatiguing or annoying the recipient into unsubscribing or hitting spam, this is the sort of thing we mean.

Three emails (to the same email address) in four minutes might be a bit much.

If you can’t combine the content you want to send into a single personalized email, maybe spread deliveries out a bit? Or even not send all of it, perhaps.

The post Tempo appeared first on Word to the Wise.

Filters evolving

0
0

I started writing this blog post while sitting on a conference call with a bunch of senders discussing some industry wide problems folks are having with delivery. Of course the issue of Microsoft comes up. A lot of senders are struggling with reaching the inbox there and no one has any real, clear guidance on how to resolve it. And the MS employees who regularly answer questions and help folks have been quiet during this time.

In some ways the current situation with Microsoft reminds me of what most deliverability was like a decade ago. Receivers were consistently making changes and they weren’t interacting with senders. There weren’t FBLs really. There weren’t postmaster pages. The reason knowing someone at an ISP was so important was because there was no other way to get information about blocking.

These days, we have a lot more institutional knowledge in the industry. The ISPs realized it was better to invest in infrastructure so senders could resolve issues without having to know the right person. Thus we ended up with postmaster pages and a proliferation of FBLs and best practices and collaboration between senders and receivers and the whole industry benefited.

It is challenging to attempt to troubleshoot deliverability without the benefit of having a contact inside ISPs. But it is absolutely possible. Many ISP folks have moved on over the years; in many cases due to layoffs or having their positions eliminated. The result is ISPs where there often isn’t anyone to talk to about filters.

The lack of contacts doesn’t mean there’s no one there and working. For instance, in the conference call one person asked if we thought Microsoft was going to fix their systems or if this is the new normal. I think both things are actually true. I think Microsoft is discovering all sorts of interesting things about their mail system code now that it’s under full load. I think they’re addressing issues as they come up and as fast as they can. I also think this is some level of a new normal. These are modern filters that implement the lessons learned over the past 20 years of spam filtering without the corresponding cruft.

Overall, I do think we’re in a period of accelerating filter evolution. Address filtering problems has always been a moving target, but we’ve usually been building on known information. Now, we’re kinda starting over. I don’t have a crystal ball and I don’t know exactly what the future will bring. But I think the world of deliverability is going to get challenging again.

 

The post Filters evolving appeared first on Word to the Wise.


What kind of mail do filters target?

0
0

All to often we think of filters as a linear scale. There’s blocking on one end, and there’s an inbox on the other. Every email falls somewhere on that line.

Makes sense, right? Bad mail is blocked, good mail goes to the inbox. The bulk folder exists for mail that’s not bad enough to block, but isn’t good enough to go to the inbox.

Once we get to that model, we can think of filters as just different tolerances for what is bad and good. Using the same model, we can see aggressive filters block more mail and send more mail to bulk, while letting less into the inbox. There are also permissive filters that block very little mail and send most mail to the inbox.

That’s a somewhat useful model, but it doesn’t really capture the full complexity of filters. There isn’t just good mail and bad mail. Mail isn’t simply solicited or unsolicited. Filters take into account any number of factors before deciding what to do with mail.

What kinds of factors?

There are five broad questions I think about when guiding clients through their email programs.

  • Is the mail safe?
  • Is the mail solicited?
  • Is the mail targeted?
  • Is the mail wanted?
  • Is the mail productive?

Different filters have different weights for the categories. Those weights explain why delivery can range so widely across domains and email providers.

Let’s look at each set of factors and talk about who might care more about those factors than others.

Is it safe?

Does the message contain malware, phishing, anything that could harm the recipient’s computer or the network as a whole? These filters are widespread and heavily weighted by most people. Safe doesn’t typically come into it for legitimate mail, but the filters are still there and still sniff at our mail.

Is it solicited?

Alternatively, did the user ask to receive mail from the sender? Many blocklists, including Spamhaus, specifically set out to block unsolicited email. They don’t really care about what the email is. They simply want to make sure that the recipients are receiving mail they asked for.

Confirmed opt-in is a way to ensure that mail is solicited. The folks behind many of the blocklists simply want users to receive mail they asked for. Senders who can demonstrate the mail is solicited get removed from the list.

At ISPs, solicited is somewhat important, but the signs of solicited mail overlap with signs of wanted mail. When ISPs measure unknown users and complaints, part of what they’re trying to determine is if the mail is solicited by their user.

Is it targeted?

Does the user understand why they’re receiving the mail? As a small business owner, I get a lot of targeted email. Random companies buy addresses and target me as someone who might want their service. The mail is targeted, so some filters, particularly those at ISPs, might not block or spam folder the mail.

But just because mail is targeted doesn’t mean the user wants it.

Is it wanted?

Does the user want the mail? Sometimes they do, sometimes they don’t. The big webmail providers (Oath, Microsoft, Gmail) heavily weight wanted. They don’t care so much if the message is solicited or targeted, although both things will increase the likelihood that the mail is wanted. At these ISPs, filters really focus on signs that the user is engaged with the message as part of the delivery process. Wanted mail gets into the inbox, unwanted mail not so much.

But just because the mail is wanted doesn’t mean it will make it to the inbox.

Is it productive?

This filter only really comes into effect when we’re talking about mailing into businesses. Email is a tool for businesses and they often want employees to be working while at work. Even if an employee solicits and email a business might decide it’s not productive for the business and they block that source of email. Likewise, businesses will block targeted and wanted messages simply because they’re unproductive.

What’s it all mean?

Effectively addressing delivery problems means understanding why a message isn’t reaching the inbox. Improving engagement isn’t going to help senders reach employee mailboxes if the mail is unproductive. Better targeting won’t help if the block is due to the mail being unsolicited. Using confirmed opt-in won’t magically get malware into the inbox.

It used to be that deliverability recommendations would work across the range of filters. Mail that made it to the inbox at an ISP like Gmail was likely to make it into the inbox almost anywhere. But as Gmail (and Oath and Microsoft) focus more and more on custom delivery for every recipient, recommendations that work there aren’t always going to work elsewhere.

Reaching the inbox outside of webmail providers means taking a lot more into account than just if the recipient is engaged with your mail.

 

The post What kind of mail do filters target? appeared first on Word to the Wise.

What does mitigation really mean?

0
0

It is a regular occurrence that senders ask filters and ISPs for mitigation. But there seems to be some confusion as to what mitigation really means. I regularly hear from senders who seem to think that once they’ve asked for mitigation that they don’t have to worry about filtering or blocking at that ISP for a while. They’re surprised when a few weeks or even days after they asked for mitigation their mail is, one again, blocked or in the bulk folder.

The words What Makes You Special on a badge, asking the question of what characteristics set you apart as an individual as different, unique, distinguished or better than the rest.

What is mitigation?

Think of mitigation as a flag that tells spam filters to ignore the history for an IP or domain. The history isn’t deleted or removed, it’s still there. But the “start date” is moved to the mitigation date. If I am a sender that’s been using an IP for a few years and I have a few bad months of sends in the middle, I can ask the ISP to mitigate the effect of those bad months on my reputation. The sender starts over fresh, with none of the bad history.

Mitigation is not a get out of jail free card.

Mitigation is not a get out of spam folder free card. This is not something offered to senders who have a poor history. It’s primarily intended for senders who are normally good senders but had some rough sends. The intention behind mitigation is to give senders a way to get out of the spam folder after they’ve fixed their problems. The infamous MS response “we see no problem with your delivery” in respect to spam foldering means exactly that according to their numbers, mail should be delivered to the spam folder.

Mitigation is not automatic.

In most cases mitigation is handled by a human being, that is following policy established by their employer. Real people review the internal data and dashboards and make a decision based on that review. Senders who have a long history of marginal mail are less likely to receive mitigation. The corollary is that senders who have a history of decent mail but a few bad sends are very likely to receive mitigation.

Mitigation requires plausibility.

Email delivery requires cooperation among senders and receivers. Mitigation requires trust on the part of the ISP, and every ISP rep has multiple stories of spammers who abused that trust. Senders who demonstrate they’re acting in good faith, by making receiver visible changes before requesting mitigation, are much more likely to receive mitigation. Repeatedly asking for mitigation decreases the chances of it being granted. Remember, mitigation doesn’t erase data, it simply resets the start time for analysis. The person handling mitigation can see that it was granted and nothing changed. It’s not a plausible request the second, or third, or fourth time.

Mitigation is an exception.

Asking for mitigation is a normalized pathway, but it’s not normal. Senders make the mistake of thinking if they got mitigation once, they just have to ask again. That mistake leads them to ask for mitigation without changing anything about their sends before or after the mitigation. As a result, they discover their mail is back in the spam folder. This also leads to mitigation not being granted a second or third time.

Don’t rely on mitigation.

No sender should rely on mitigation to get to the inbox. Instead, senders should focus on the fundamentals of good delivery: sending mail people ask for and expect. Everything else is rearranging deck chairs on the Titanic.

 

 

 

 

 

The post What does mitigation really mean? appeared first on Word to the Wise.





Latest Images