What Is A/B Testing?

You may not be able to argue with statistics, but you should always be able to act on them. Even as you’re reading these words, your business is generating data that is worthy of analysis.

Your website is converting visitors into active prospects . . . or not.

Your social media posts are driving traffic back to your site . . . or not.

Your email newsletter signup box is growing your subscriber list . . . or not.

Of course, no one wants to be in the “or not,” category, which is why you need to make A/B testing a standard part of everything you do.

A/B testing in a nutshell

With scarce resources and never enough time, small and medium-sized businesses often feel the need to develop a strategy or course of action and then stick to it, no matter what. That might make sense when you’re developing a product or service, but it’s a risky way to pursue the marketing of your products and services.

Marketing often involves elements of creativity in terms of the copy you write, the graphics or artwork you include on an ad or landing page, or even videos you shoot. Creative work is always a little subjective. You make what you think will work, often after scrapping a number of your initial ideas. A/B testing recognizes the fact that sometimes, the ideas or actions you don’t pursue might actually be the better choice.

When you A/B test, you expose your audience (or even just segments of it) to multiple variations of creative work. You change up the frequency at which you publish content, the nature of how you promote something or other aspects of your marketing plan.

The results of A/B tests can often help you see what’s really working among your target market, and what’s not. It allows you to refine your strategy so that you don’t waste your budget and maximize the return on the investment you’re making.

There are plenty of technologies, like marketing automation, and tools to assist with A/B testing, but these are the basic principles you should know to get started:

1. Pick the metric an A/B test could influence

Let’s say you want to use a landing page to collect leads for your sales team. The landing page might have a form site visitors have to fill out in order to download some valuable content, like an eBook or buyer’s guide. The design of that landing page will obviously be key to your success, but before you A/B test different versions of it, think of what you’ll be measuring in the test.

Do you want to simply track how many people visit the page, or how many people fill out the form and download? The first metric might be affected not only by the design but the keywords you’ve included that get found in search results. The latter metric might have more to do with the number of fields in the form, as well as how the asset is described.

Create an A/B test knowing in advance if you’re primarily aiming for awareness, engagement, indications of purchase intent or some combination of metrics.

2. Pick your test sample

It can be confusing if your entire audience finds themselves looking at different versions of the same marketing assets or creative campaign materials. The usual route is to create a subset of your audience that will give you statistically relevant data you can use to make a decision about your market as a whole.

The exact size of your sample might depend on your audience, but let’s use an email newsletter as an example. You might want to do an A/B test of different subject lines to see which one caused more people to open up the message and click through to any links you’ve included.

An A/B test could involve sending out one subject line to five per cent of your distribution list, and the other could go to another five per cent. Even if one of the subject lines falls flat, you still have a good sense of what will likely resonate for the remaining 90 per cent of your list.

3. Pick your testing time frame

Some A/B test results will be apparent right away, but others might require a little more patience.

The impact of a new version of an online ad, for instance, might need to factor in the time it has been available online and the number of sites on which you’ve placed it.

If you’re A/B testing content for social media, you might need to try posting multiple times, or at different times of the day, to truly gauge the effectiveness of different versions.

Much like your sample size, the time frame can be dependent on a number of factors, but generally you need to be able to gather enough data to feel confident in choosing your “winner.”

Conclusion: Turn your A/B test results into better business results

Although they’re called A/B tests, you might test more than simply two variables. Use as many as you need based on the goal you’re trying to reach.

Once you have your test results, meanwhile, the next step might need to go beyond settling for whichever variation performed the best. You might want to continue tweaking the “winner” based on language, imagery or many other different factors. In that sense, the A/B test could be the first of many as you optimize your marketing.

Not all A/B tests will be based on a marketing campaign, either. You should also use them for marketing assets that tend to be created for long-term use, such as product pages or even your “About Us” and “Contact Us” pages on your website.

Some A/B tests will also involve looking at whether you stop doing something. For example, you could A/B test what happens when you don’t run an ad during a specific period, or if you turn off a social promotion in favour of sharing other kinds of content.

Whatever the approach, A/B testing will make you a more data-driven marketer — in other words, the kind of marketer who achieves the most ambitious business objectives.