New, exclusively for members
Free Personalized Learning Plan
Find the resources that are right for you with a personalized learning planGet started
There have been a couple of heated debates I've been part of in the email community just recently, one about open rates and another about subject line length. In this post I'm reflecting on the open rate debate. I've added an update to the original post showing more proof that open rate fixation can be misleading.
It may surprise you that the community is very split on the value of the open rate metric, seemingly, a cornerstone metric since the dawn of email marketing. It's a metric cited in every email marketing book and on every mail marketing course... So why is its value in doubt? Ken Magill reported on some of the open rate debate.
Let me summarise some of the reasons why open rates are not important.
I've recently published some research for emailvision reviewing whether open rates are a good predictor of success in email marketing across 50 million retail emails from 196 campaigns. The headline finding is...
Open rates wrongly predict success 53% of the time
As you might expect, there is a much closer correlation between clickthrough rate and open rate. This graph from the research shows that the open rate is relatively unchanging; it typically varies by just 12%. So if the open rate average is 20% that means it varies up to 22.4%. However conversion rates vary hugely in comparison, by some 70%. If average conversion rate is 0.2% then it varies up to 0.34%. To understand campaign performance the conversion rate is a must.
Why can a higher open rate mislead and deliver less business? Take an example, I talked to someone recently who was delighted they had increased their open and click rates. I asked how. The answer was they removed people who hadn't opened recently. I asked if they had conversion and revenue figures to compare. I wasn't surprised when I didn't get an answer...
The five different cases below illustrate many of the problems with open rates.
List size 10,000
Number of people who open 2,000
Open rate 20%
A common percentage of inactives is 40% of the base. Let's remove those 4000 people from campaigns.
List size 6,000
Number of people who open 2,000
Open rate 33%
That's a 50%+ increase in open rate. Will it have resulted in 50% more revenue? Absolutely not. In fact most likely the revenue will decrease, because every month a percentage of the apparently inactive customers do come back again.
The open rate metric and increase in this case does not represent business success.
Campaign open rates are sometimes averaged over the month for reporting. For example these three campaigns:
Campaign A 60%
Campaign B 20%
Campaign C 10%
This gives the monthly open rate, calculated as the average of open rates, of 30% = (60+20+10)/3.
Let's add real numbers to the above figures
Total emails sent 10,000 with total opens 1600, giving an open rate over all campaigns of 16%. The commonly used average of opens, at 30%, is almost double the real value.
Two campaigns with identical open rates. Which campaign was more successful? Ok, so you're getting wise and want to know the send size before you decide? That's fair, here are more campaign metrics.
|Delivered||Open rate||Click rate||Conversion rate|
They still look the same. However, campaign Y is more successful. Because the average order value for campaign Y was double that of Campaign X.
Customer segment A receives a campaign with open rate 20% and click rate 5%.
Customer segment B receive two campaigns in the same period and because of email increased marketing pressure the open rate drops to 15% and click rate to 4%.
Which was more successful?
If the list size is 10,000, then the number of opens for segment A was 2000 and for segment B was 3000 across the two campaigns. Making campaigns to segment B more successful, although the campaign open rate is lower.
Surely the open rate is a useful measure of the subject line quality?
Well, not really here either. Take the case where one subject line gets lower opens and higher clicks than the other. The better subject line is the one with higher clicks and lower open rate. Fewer people may have opened but the ones that did open were the right ones, the right target for the offer and call to action.
It would be even better to evaluate the subject line on conversions, however test cell sample sizes don't normally permit that.
If not open rate then what?
To sum it up, the better metrics to focus on are:
Notice how these metrics consider money and customers not campaigns. Measure campaigns to understand what campaigning strategies work and measure customers too, as customers are more fundamental to business health.
What other metrics, whether new ones or old ones should have time called on them?
Next time around I'm going to look at the debate around subject line length and explain just what the right length is for a subject line.
By Tim Watson
Tim Watson, from consultancy Zettasphere, is the Smart Insights expert commentator on email marketing. He is an independent email marketing consultant providing strategic guidance, to deliver improved campaign results. A member of the DMA Email Council and chair of the DMA Email Best Practice hub. He actively promotes email and frequently speaks on the subject and how to improve use of the channel. Connect with Tim via LinkedIn or Twitter.
Start the discussion on our community and social networks
Recommended Blog Posts
Popular Blog Posts
Statistics on consumer mobile usage and adoption to inform your mobile marketing strategy mobile site design and app development “Mobile to overtake fixed Internet access by 2014” was the huge headline summarising the bold prediction from 2008 by Mary Meeker, an …..
Landing page examples and best practice advice Discussion of web design in companies who don’t know the power of landing pages still often focuses on the home page. But savvy companies know that custom landing pages are essential to maximise conversion …..
Use the RACE Planning System to get ahead in your digital marketing The first edition of my book Internet Marketing: Strategy, Planning and Implementation from 2001 included a popular template for creating what we then called an Internet Marketing Plan. Today, …..