Conversion Rate Optimization: How to A/B Test with Success

The issue you're facing is that you need more conversions, whatever that means to you and your business. Well, you’ve come to the right place!

In this post, we’ll dive deep into what conversion rate optimization is and how it can help your bottom line.

What Is Conversion Rate Optimization?

Conversion Rate Optimization (also referred to as CRO), simply put, is the process of increasing the percentage of people who take a desired action. In the digital world, this action is typically the end-all be-all conversion event that takes place either on site or in digital advertising. The selection of this conversion event is extremely important when configuring and conducting tests, which we’ll discuss in more detail in a later step.

One of the most common forms of Conversion Rate Optimization is achieved with A/B Testing. In fact, according to one study 58% of companies are using A/B testing as a conversion rate optimization catalyst. So, then this begs another question. What is A/B Testing?

What Is A/B Testing?

A/B testing is the process of showing 2 different experiences to site visitors and measuring the impact of that change on a target value (in other words, the conversion event). Even the smallest change can have the biggest impact. In fact, according to one example in the Harvard Business Review, a single Microsoft Bing A/B test changed the way ad headlines were displayed and added $100M (or 12%) revenue in one year.

While we can’t promise those results, our Digital Data Analytics team approaches A/B testing in two different ways:

On-site A/B Testing

• On-site A/B testing is, well just what it sounds like, changing web elements on the site that we control once a user reaches the site and measuring which experience resulted in higher conversions.
• On-site changes can span from extremely simple changes like changing the color of a CTA, to much more complex changes like adding a cart hover functionality to an ecommerce shop.

Off-site A/B Testing

• Off-site A/B testing focuses on making changes to digital marketing creative assets or copy to see which experiences bring more users to the website. In other words, the conversion action here will be, 9 times out of 10, a click on the actual advertisement. This type of A/B test is done in conjunction with digital marketing teams.

How to Conduct An A/B Test

Using the above image as an example. Regardless of the toolset you use to run your A/B tests (think Google Optimize, Adobe Target, or any CMS worth their salt) there's a common cycle that occurs throughout all A/B tests. We've outlined and defined the 7 steps that make up that cycle below.

Define the Website Conversion Event

• What is it that we want to gain more of from the A/B tests? Maybe it’s transactions for an ecommerce website and leads generated for a B2B site?
• Make sure the conversion events are tagged from an analytics perspective.

Brainstorm A/B Test Ideas

• Come together as a team and hypothesize different test options that could potentially have an impact on the selected conversion event.
• Optionally, if your website is already well tagged from an analytics perspective, you can analyze the behavioral data from your users and use data to drive your hypothesis!

On-site Change Design Comps

• Depending on the level of effort for the test, it may be smart to get help from a design team to mockup comps of what the test will look like. We have a very talented design team that helps us in our A/B test workstreams.

A/B Test Development

• Depending on the difficulty of the test, it may be wise to get help from a development team to implement the A/B tests. We have a very talented development team on staff that helps us with our more complex A/B test workstreams.

A/B Test Implementation

• Finally, we can run our tests on the website and see real-time results!
• At this point, it is best practice to let the test run for at least 2 weeks and wait until the results are statistically significant. Statistical significance, in the sense of A/B testing, is the probability that the on-site changes have a direct impact on the conversion event with very little chance that it is random. In other words, our test worked!

Analyze Results

• Once the test is complete (winner or loser) we can turn it off and analyze the results.
• Part of our regular workstream is doing an impact analysis. This allows us to say with certain that our A/B test had X effect on the conversion action and here’s what that means for your business. If the site is an ecommerce shop, it’s best to communicate this impact in terms of dollars. If it’s a B2B lead generation site, leads should be the primary metric for communication.

Implement Changes

• If the test is a winner, it’s best to get the on-site change implemented immediately so we can start reaping the benefits of all our hard work!

If the test is a winner, it’s best to get the on-site change implemented immediately so we can start reaping the benefits of all our hard work!