What is A/B testing and why should you A/B test?

You’ve probably heard of A/B testing before. It’s a common web analytics approach when you need to refine a process or asset.

In marketing, A/B testing can result in better advertising creative, email newsletters, landing pages, and more. But first you need to understand how it works, why it’s important, and what the best practices have become.

We’re going to guide you through the process of A/B testing your marketing materials from start to finish. That way, when you’re ready to design your own materials, you won’t have to start from square one.

More importantly, you’ll know how to conduct accurate testing to find the best headlines, images, photographs, copy, and more. After all, you want your marketing materials to work for you — not against you.

What Is A/B Testing?

A/B testing is the process of comparing two versions of the same marketing material, such as a sales page or a Facebook Ad. The two versions are almost identical except for one variable.

It might sound like a lengthy process, but it can yield valuable results. Do people prefer red or black CTA buttons? Do they like Headline A or Headline B? Which image results in more clicks on the CTA?

You’d be surprised how much impact a little detail can have on conversion rates and other marketing facets. Many large companies have conducted extensive studies using A/B tests, and the results have caused them to make huge changes to their marketing and advertising approaches.

For instance, maybe the copywriting on your email newsletters has resulted in a high open rate, but a low CTR. In that case, you might want to change up the copy a little and A/B test the variations.

How Does A/B Testing Work? 

During A/B testing, an entrepreneur or marketer can compare engagement rates between two pieces of nearly identical content. This means that A/B testing relies on a specific goal.

Let’s take Facebook Ads, for instance. Maybe you want to improve click-through rates on your Facebook Ads that focus on retargeting customers with a discount.

In this case, CTR is the engagement you’re tracking. You want to know how many people click through on each Ad so you can decide which one performed better.

The engagement could be something else entirely. You could track open rates on your emails, for instance, or conversions on your sales pages. It all depends on the metric that matters most right now for your business.

After you complete your first A/B test, you can begin the process over again. This time, you might change a different variable or focus on a different metric.

Why Should You Conduct A/B Tests? 

Many aspects of marketing feel like a shot in the dark. You put something out there — whether it’s a live webinar, a blog post, or a YouTube video — and hope it helps you boost your sales in some way.

Sometimes it works. Sometimes it doesn’t.

However, the best marketers know that they can rely on data to refine the process. They don’t want to bank on that shot in the dark. They want to make decisions based on empirical evidence.

It’s true that other companies have posted case studies and surveys that reveal their own results. The problem is that your audience isn’t the same as theirs.

Maybe your audience prefers a cooler color scheme, while another company’s audience goes for bold, warm colors. If you make decisions based on their data, you won’t get as many conversions or other positive responses.

What Elements Can You A/B Test?

Technically, you can A/B test anything you like. However, certain elements in a piece of marketing often prove more impactful than others.

Some of the most popular elements to A/B test include the following:

• Headlines
• CTAs
• Images
• Content length
• Content depth
• Offers
• Subject lines (for email)

Each of these elements can have a huge impact on whether or not your visitors, viewers, or subscribers convert.

For instance, you might run a series of A/B tests on your drip email campaign. In order, you might test:

• Subject line
• Offer
• Length of email and copy
• CTA

After you run those four A/B tests, you can figure out what combination of elements results in the most click-throughs or the most conversions.

The same goes for any other piece of marketing or advertising copy.

You can also run A/B tests on two related pieces of copy. Let’s say that your email newsletter has a CTA that leads to a landing page. You’ll want to run tests on both the email newsletter and the landing page to perfect your sales funnel.

What Are Champions, Challengers, and Variations?

Even if you’re not a professional marketer, it can help to know the lingo. When you’re researching a topic, such as A/B testing, you’ll want to understand what the experts are talking about.

There are three types of pages that entrepreneurs can test using the A/B format.

Champions are pages that have provided excellent results in the past. They’re highly effective, but you want to know if you can improve upon them even more.

Challengers are pages you want to test against your champion. You’re interested to see whether or not they produce similar or better results.

Variations, meanwhile, are different versions of your challengers. You can run lots of A/B tests consecutively, with different variations, until you find the perfect recipe for your business and audience.

Should You Start With an Existing Page?

If you’ve already been in business for a while, you likely have landing pages, sales pages, blog post CTAs, and other pieces of marketing copy and creative. In this case, you can start with an existing page for your A/B test.

The only time you don’t want to start with an existing page is when you’re getting no engagement at all. For instance, let’s say 10,000 people have received your email and zero people have clicked through to your landing page.

It’s time to start over from scratch.

Now, if you’re just launching a Knowledge Commerce business, you don’t have existing copy or creative. In this case, you’ll want to create your page from whole cloth, then create a variation for the test.

Just remember that you don’t want to test too much, too soon. If you don’t have a large audience, the results won’t provide sufficient insight to make informed decisions.

What Is Segmentation? 

Many entrepreneurs think of their audiences as some collective consciousness. It’s not. That’s why you create buyer personas and target different people with different types of copy.

“Segmentation is an aspect of A/B testing that allows you to refine your results even further. During the test, you present different champions, challengers, and variations to different segments of your target audience. #Kajabi” — Tweet this!

For instance, you might have one A/B set that’s targeting young, upwardly mobile, single women, while another is geared toward married women with kids. Those are two completely different audiences, so you’ll want to segment them for your tests.

Do you have to segment every test? Of course not. If your audience is relatively static, or if you don’t have a large following yet, conducting simpler A/B tests will work just fine.

Does A/B Testing Always Work?

The main factor to consider when running A/B test is the sample size.

It’s kind of like a scientific experiment someone might run in a lab. If the study includes 300 subjects, the results will be more accurate than if the scientific team worked with just 30 subjects.

In any test, you want to have the largest sample size possible so you can rule out aberrations in the data. Lots of things can impact a consumer’s willingness to click on a link.

For instance, maybe a consumer didn’t click through because he was late for a meeting or because she received a phone call. Interruptions and disruptions can influence people in ways that have nothing to do with your copy or creative.

If you have a small audience, realize that your results might be slightly skewed. However, you’ll still gain valuable insight into your existing audience.

What Is the A/B Testing Process?

An A/B test always starts with two versions of the same piece of marketing, whether it’s an email, a landing page, or something else. You’ll change a single variable so you can pit one against the other.

After you’ve created your marketing materials, you release them into the wild. You send out the email, run the Facebook Ads, or push the landing pages live. For the test, you steer half your audience toward page A and the other half toward page B.

You’ll also decide on the engagement you want to measure. It could be CTR, conversions, sales, traffic, or anything else you want to measure. You can also measure multiple metrics at once as long as they don’t interfere with the test.

That’s the simplified version. But let’s look at a breakdown of each step so you can follow it to a T when you run your own tests.

1. Examine Existing Data

Assuming you already have your business up and running, it’s time to take a look at your data. Fortunately, Kajabi makes it easy with built-in analytics tools that help you track whatever metrics you wish.

Figure out what aspects of your marketing strategy might need work. Where are you seeing the lowest conversion rates? What about the highest?

Ideally, you want to find something that needs improvement. For instance, are you experiencing extraordinarily low open rates on your emails? Maybe it’s time to A/B test the subject lines to see if you can’t bump up those numbers a bit.

Looking at your existing data will help you spend your time more wisely on testing. Otherwise, you’re just taking a wild guess.

If you don’t have any existing data, you might want to collect some before you start your tests. You can go in blind, but you won’t have a baseline from which to begin work.

2. Set Specific, Measurable Goals

No scientific experiment or test can work if you don’t have goals to begin with. The goals need to be specific as well as measurable so you can track your progress.

In terms of specificity, you want a precise number and deadline. For instance, maybe you want to increase email open rates by 3 percent over six weeks. That gives you a number to hit (3 percent over the current open rates) and a deadline (six weeks from the start of the test).

How do you decide on your goals? Look at past patterns. You can also research similar companies to your own to find out if they have any data that might help reveal what your goals should be.

Remember, though, that goals should be attainable. Let’s say that you’re currently selling $200 worth of Knowledge Commerce products for month. If you set a goal to hit $10,000 in revenue in two weeks, you’re likely to be disappointed.

3. Generate a Hypothesis

Here’s where you get to put in a little guesswork. What do you think will happen at the end of your test?

You already have a goal you want to meet, but do you think that the variations you’re running for the A/B test will yield those results? Why or why not?

For instance, let’s say that you want to A/B test three aspects of your confirmation emails: subject line, headline, and CTA.

Based on previous experience and past data, which do you think will have the biggest impact on open or conversion rates? Which do you think won’t matter much at all?

The point of a hypothesis is to predict results. You might not be right, but your mistake will be informative. Plus, you’ll get better at predicting results the more you A/B test.

4. Create Variations

It’s time to create the copy and creative. You want to run identical versions of both for your A/B test, but you’ll change just one variable.

Many entrepreneurs think they can shortcut the system by testing multiple variables at the same time. They run tests by changing the headline, CTA, and image, for example.

Don’t fall into that trap. It won’t work.

Why? Because your data will be flawed. You’ll have no idea whether your engagement rates differed because of the headline, CTA, or image. It could have been one or all three.

Always test one variable at a time. After running your initial test, you can run a subsequent test to compare a different variable.

5. Run the Experiment

At this point, you’re ready to run the experiment. Choose whatever A/B testing tool you prefer (we’ll go into more detail later on) and run the test for the duration you’ve selected (more on that later, too).

You might feel tempted to cut the test short if you see major differences between the two variations. Resist the urge. You can get excited, but let the full test run its course.

6. Analyze the Results

After the test concludes, take a good, long look at the data. What can you learn from it? What’s your next step?

A great A/B test run will show major differences between the two variations. If you have a large enough sample size, you can extrapolate from the data that one version outperformed the other because of the variable you changed.

Subsequent A/B tests can include the “winning” variable. You’ll then change another element to see what impact it has on engagement levels.

How Are A/B Testing & SEO Linked?

We’ve mentioned before that duplicate content can present a problem when it comes to SEO. Google penalizes websites that publish duplicate content (content that exists elsewhere on the web) because it’s considered less valuable.

Of course, A/B testing is all about duplicate content. As we said above, you don’t want to change more than one variable.

When it comes to email and ads, SEO isn’t much of a factor. However, if you’re A/B testing landing pages, sales pages, or other static pages on your site, you don’t want to make the Google beast angry.

So how do you conduct A/B testing and make sure you don’t harm your SEO in the process?

It’s actually relatively simple. If you follow the tips below, you won’t have to worry about any penalties.

Don’t Use Cloaking

Cloaking sounds a little dark and insidious, doesn’t it? Google thinks so, too.

When you cloak a page, you try to show Google and other search engines different content than a typical user would see upon landing on the page. It’s a big no-no in the SEO world.

Most people use cloaking by segmenting content based on user-based IP addresses. If that doesn’t sound familiar, you probably don’t have to worry about it.

The important thing to take away is that you shouldn't try to hoodwink the search engines. You will get caught — and penalized. It’s better to stay above board and present content in a search engine-friendly way.

Implement the Canonical Tag

Another way to improve your A/B testing in terms of SEO is to use the rel=”canonical” tag. It’s an HTML tag that tells the search engine which version of a web page is the “master” or main page.

For instance, we talked about champion pages above. If you already have a landing page that’s working well, it’s a champion. Add the canonical tag to that page so search engines know it’s the primary page.

When you publish variations, Google won’t consider those pages to be duplicates because of the canonical tag. Do this any time your A/B test involves multiple URLs.

Use 302 Redirects Instead Of 301s

It’s common to use a 301 redirect when you want to permanently steer visitors to a certain page to another page. Maybe you’ve replaced the content with something new at a different URL, for example, and it’s going to stay that way forever.

However, 301 redirects aren’t useful for A/B testing because the variations are temporary. You’ll eventually choose the version you like best and deindex the variation.

To avoid annoying Google and potentially hurting your SEO, use 302 redirects for A/B testing purposes. This tells Google that you’re temporarily redirecting users to a different URL, but that you’ll eventually remove the redirect.

Keep Your A/B Tests as Short as Possible

The potential SEO ramifications of A/B testing get more probably the longer you run your tests. While you don’t want to sacrifice data in a too-short test, neither do you want the experiment to run longer than necessary.

How Long Should You A/B Test Each Element?

Speaking of duration, your A/B test should ideally run until you have a large enough sample size to qualify for statistical accuracy. In other words, your test shouldn’t conclude until you have enough data from which to extrapolate accurate results.

That’s a mouthful, so let’s break it down.

When you run an A/B test, your sample size is dependent on the number of people who see each variation. If you have a small sample size, the data will be considered statistically irrelevant.

That’s bad because you might make decisions based on faulty data.

Some companies can run A/B tests for just a few days and get lots of meaty data. Others need several weeks. The tool you use for testing can help you decide how long you’ll need to run the experiment for maximum statistical accuracy.

What Is the Best Way to Analyze Your A/B Test Results? 

When your A/B test concludes, use the best possible tool to analyze your data. You’ll want to consider the goal you set, the hypothesis you made, and the specific metric you wanted to track.

You can use Kajabi’s built-in tools to track data on metrics like traffic and conversions. You can also use third-party services if you’re running a more formal A/B test.

It all depends on how much money you want to spend and what you’re testing.

A/B Testing Examples

Let’s say that you teach online courses on fitness. You’ve decided that you’re not getting the engagement you want, so you turn to A/B testing to improve your marketing results.

Maybe you want to start with the landing page you use to get people to sign up for your email list. It includes a headline, a hero image, a list of benefits of the email list, and a CTA.

You might start with the headline since it’s the first thing people see. You want to know what type of headline will keep people on the page — and encourage them to scroll down.

Maybe your existing headline is in the form of a question. You decide to test it against a statement-based headline.

From there, you could A/B test the hero image and the CTA. Based on the data, you’ll create the ideal landing page.

Or maybe you want to test different Facebook Ads for your latest online course. You’re teaching people how to lose weight with low-impact fitness, so you decide to A/B test two different pieces of creative.

The first shows an extremely fit man or woman working out at the gym. The second shows a less-fit man or woman struggling to work out.

You want to know whether the positive image or the negative impact results in more click-throughs.

Your A/B tests can follow similar patterns. You’re looking for the elements that have the biggest impact on the metrics you want to improve.

Best A/B Testing Tools

There are lots of A/B testing tools out there. As we mentioned, Kajabi’s built-in analytics are a great place to start.

You can also use third-party tools to help refine your A/B test. Optimizely, for instance, is one of the most popular and powerful tools on the market.

Other tools to explore could include:

• KISSmetrics
• Unbounce
• Google Analytics
• Crazy Egg

Use Kajabi to Turn Your Knowledge and Content Into Products You Can Sell

Are you ready to start your own business and sell your knowledge? We can help. Kajabi has provided a platform that allows anyone to sell their knowledge and many money from home.

We’re offering a 14-day free trial so you can test-drive the platform and get used to the software. As soon as you sign up, you can start creating your first online course or other digital product.

Conclusion

A/B testing is one of the most effective ways to refine your marketing strategy. It’s simple to conduct and produces excellent results.

As long as you’re familiar with the A/B testing process and aware of the potential ramifications on your website’s SEO, you’ll do fine.

You start by checking out your existing data. Then you set a goal, build a hypothesis, and create variations. After you run the experiment, you’ll analyze results and draw conclusions.

‍
Find more blog posts by category:

Create Your Product

Build Your Business

Grow Your Business

Kajabi News

Kajabi Hero Stories