Testing is the best way to find out what makes your customers tick, but if you market across multiple digital channels, you must come up with a testing protocol for each channel: email, web, social media, retargeting, SMS etc.
Or do you?
And even though you already know that email is the hub of your digital marketing programme because you can use email addresses to identify and communicate with your customers across all your marketing channels, most email marketers are not performing the basics and testing their emails as stated in latest report from GetResponse “Email Marketing & Beyond: Global Industry Benchmarks 2017”.
Testing via email can drive your multichannel testing programme and deliver insights across your entire customer database, thereby not only saving you time and money in driving traffic but also providing increased conversions sooner. After all, your email database is representative of your target market – your prospects, your first-time purchasers, your regular customers, your loyal customers as well as your lapsing and lapsed customers.
This is Holistic Testing – a new testing methodology by Holistic Email Marketing - which moves beyond single-channel and basic A/B testing to deliver a more powerful set of customer insights that inform, improve and direct your entire marketing programme.
Most of the testing that email marketers do today is "ad hoc" testing: one-off, irregular testing of factors that uses no scientific methodology or hypothesis and doesn't build on previous testing results. Email marketers perform these tests with the aim of determining which version delivers a better result. However, in many cases because of the lack of scientific approach to performing the test, the results are not valid and disbelief in testing sets in.
An example of this is a test that has been set up to determine which subject line delivers the highest clickthroughs. No hypothesis has been defined.
Control: Our sale is on now! Get 15% off today!
Variant: Our sale is on today! Get 15% off now!
In contrast, Holistic Testing comprises a series of regular tests, each of which uses a hypothesis and scientific methodology to guide the research and improve its validity. Additionally, Holistic Testing aims not just to learn which version "won" but also to discover long term insights about the audience that you can roll out to other channels.
Hypothesis: Your statement of the presumed problem, your proposed solution and what you expect your test will reveal. Your results will either support or refute your hypothesis. An example hypothesis could be “Loss aversion copy is a stronger motivator towards conversions than benefit-led copy because people hate losing out more than they enjoy benefiting.”
Your hypothesis helps you choose the right factors to test. Most testing focuses on a single element, but you can test multiple factors that help to test your hypothesis, such as determining whether loss aversion or benefit statements drive more conversions.
For the above hypothesis, you could test subject line, copy and CTA simultaneously. You're still only testing one thing – your hypothesis – but you're using multiple factors working together to test that hypotheses. This is a major difference from normal testing.
Using the same Control Subject line and the above hypothesis, this is how a Holistic test may look, testing multiple factors based upon the same hypothesis:
Subject Line:
Control: Our sale is on now! Get 15% off today!
Variant: 15% off today only. Don’t miss out!
CTA:
Control: Get 15% off >>
Variant: Don’t miss out, act now >>
Scientific Methodology: How you will conduct your test and measure the results. It beats just plugging random variables into your testing platform and seeing what happens. Having these guidelines makes your results less vulnerable to chance or unknown factors.
Now that you have defined which is the winning motivator – benefits or loss aversion - you can repeat this test using email a couple more times to ensure the results weren’t due to an anomaly. If all the tests performed support the same findings, then you can roll out the insights to other channels such as your landing pages, PPC ads, Social Media etc.
You'll be better equipped to persuade management and other marketing teams that email is essential to assisting other channels, not just converting customers (where we already know it's awesome).
A carefully constructed plan of different hypotheses will give you reams of insights and help guide planning for future campaigns.
Suppose your email tests show that your first-time buyers respond more enthusiastically to free shipping without a purchase limit, but your loyal customers – those who bought three or more times in 12 months or who belong to your loyalty club – prefer VIP access and exclusive offers.
You can apply this to your website copy and organisation, to search keywords, PPC campaigns and related landing pages, and in ads and banners, you run on third-party sites in remarketing or network campaigns. Pretty soon, you’ll find more budget is being allocated to the email marketing department as it’s turned into the main testing channel.
A simple email A/B test is easy to set up and manage in an email and will deliver concrete information that you can build on in future emails. Or, use the newly discovered insights as the starting point on your landing page to incrementally improve it further with your regular website testing tool.
One change might yield only a 1% improvement, but a series of small gains over time could add up to a revolution, and could save you ££ in driving traffic to the website and start you off in a better position to reaping the rewards sooner.
Today, even basic email messaging platforms offer basic plug-and-play testing. But, even if you must pay for testing, the money you will earn from better marketing messages will more than outweigh the cost, not just on a single campaign but throughout your marketing programme.The more you know about your customers, the better you can speak to them.
The more you know about your customers, the better you can speak to them.
Instead of planning a campaign on what your instinct tells you, you can go right to your data and tell your team, "Let's use emotional language for our Mothers' Day promotion across all channels, because our testing in email shows it's more effective than humour."
You'll waste less time and money on running tests via slower channels which rely on additional budget to drive traffic as well as recoup more money because your emails are more appealing, relevant and communicating the same message (albeit potentially different wording and creative), meaning no disconnects. That's a win-win all around.
Maybe you learned about testing when you used it on your website's homepage, landing pages or product pages, and you dread the prospect of having to wait a month or longer until you get statistically significant responses.
Email testing is different because you're already testing your target audience. You can get answers almost immediately, and certainly within a day or two after you send a campaign. Email is a push channel – you don't have to wait for your customers to find you through search or drive them to the site or social media platform with paid traffic.
This means that you start to benefit by having increased conversions sooner rather than later.
Below are three example email testing hypotheses that you can apply in other channels, including your homepage, landing pages, products pages, search, PPC ads, banner ads and social media.
Factors tested: Subject line, CTA, title, copy, landing page.
The result can be applied to headlines on the website, blogs, PPC ads, banner ads, social media and retargeting ads.
Factors tested: Subject line, CTA, title, copy, landing page.
What a fantastic discovery this would be! Once assured that this was a consistent result across multiple email tests, you could apply this finding to your loyalty programmes.
Factor tested: Image on email and landing page
Again, you can roll out the results of this email test to your homepage and product pages as well as in banner ads, social media, retargeting ads as well as print ads.
These hypotheses will need multiple tests to ensure the results are not anomalies. This is their beauty. None of them are bound to any specific wording because you are testing motivation, which has been defined within the hypothesis.
Regular and systematic testing will deliver the greatest insights into your audience and help you understand what works best for your email programme. Going for long-term gains instead of short-term results gives you a solid foundation for consistent performance and incremental innovation.
Rather, it's part of a cycle of continuous learning, as you can see in the chart below. The hypothesis guides the testing, which generates data you can analyse and learn from. Applying the learnings to appropriate channels can lead to new research interests, which launch new hypotheses, and the cycle starts all over.
3. Holistic testing has one major drawback.It's addictive! Email gives you so many factors to test (way more than just the subject line) that once you get the hang of testing, you'll want to keep doing it. (But that's a good thing, so carry on.)
As the saying goes, "Always be testing!"
By Kath Pay
Kath lives and breathes email marketing and devotes her time to developing customer-centric journeys using a holistic, multi-channel approach. Kath is internationally recognized as one the leading email marketing thought leaders and is CEO of Holistic Email Marketing. You can follow Kath on Twitter and connect on LinkedIn.
Turbocharge your results with this toolkit containing 9 resources
The Business-to-business Marketing toolkit contains:
Start your Digital Marketing Plan today with our Free membership.
Recommended Blog Posts
When times are tough, content marketers must keep assessing and making changes to their content strategies to ensure their content assets are working as efficiently and effectively as they can The best content marketers in 2023 already understand that taking …..
Why you need to rethink your B2B marketing strategy to keep up with the pace of change Let’s start with addressing the status quo. For a long time, tried and tested B2B marketing strategies have been based around a linear …..
7 steps ensure top-down strategic alignment between business strategy and partner marketing Aligning partner marketing with your organization’s overall business strategy and specific strategic initiatives is essential for driving success and maximizing the value that partners bring to an organization. …..
