What Is A/B Testing?
At its core, A/B testing is exactly what it sounds like: you have two versions of an element (A and B) and a metric that defines success. To determine which version is better, you subject both versions to experimentation simultaneously. In the end, you measure which version was more successful and select that version for real-world use.
A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business. It can be used to test everything from website copy to sales emails to search ads. And the advantages A/B testing provide are enough to offset the additional time it takes.
Well-planned A/B testing can make a huge difference in the effectiveness of your marketing efforts. Narrowing down the most effective elements of a promotion, and then combining them, can obviously make your marketing efforts much more profitable and successful.
A/B testing, or split testing, is a process where we are running a simultaneous experiment between two pages to see which performs better. It is a method for validating that any new addition or change to your webpage will actually improve it’s conversion rate. An A/B test consists of creating alternative pages for a specific page, showing each of them to a predetermined percentage of visitors. In a classic A/B test, we test two versions – Version A is commonly the existing design (the “control”) and version B is the “challenger”.
An A/B test is the easiest and most common type of landing page test to conduct. Testing is also done between several versions of a page (A/B/C/D testing), but often still called A/B testing. By levels of sophistication, scientific optimization can be broken down into three categories:
A/B Split Testing
Simple testing of a page’s one element against another to see which element results in better performance.
Multivariate Testing
Testing several elements at a time. The goal is to get an idea of which elements work together on a page and play the biggest role in achieving the objective.
Experimental Design
Developing your own research method for an in-depth analysis of a specific element.
What To Test?
Your choice of what to test will obviously depend on your goals. For example, if your goal is to increase the number of sign ups, then you might test the following: length of the sign up form, types of fields in the form, display of privacy policy, “social proof,” etc. The goal of A/B testing in this case is to figure out what prevents visitors from signing up. Is the form’s length intimidating? Are visitors concerned about privacy? Or does the website do a bad job of convincing visitors to sign up? These questions can be answered one by one by testing the appropriate website elements.
Even though every A/B test is unique, certain elements are usually tested:
- The call to action’s (i.e. the button’s) wording, size, color and placement,
- Headline or product description,
- Form’s length and types of fields,
- Layout and style of website,
- Product pricing and promotional offers,
- Images on landing and product pages,
- Amount of text on the page (short vs. long).
- Create Your First A/B Test Link
The correct way to run an A/B testing experiment is to follow a scientific process. It includes the following steps:
Study your Website Data:
Use a website analytics tool such as Google Analytics, and find the problem areas in your conversion funnel. For example, you can identify the pages with the highest bounce rate. Let’s say, your homepage has an unusually high bounce rate.
Observe User Behavior:
Utilize visitor behavior analysis tools such as Heatmaps, Visitor Recordings, Form Analysis and On page Surveys, and find what is stopping the visitors from converting. For example, “The CTA button is not prominent on the home page.”
Construct a Hypothesis:
Per the insights from visitor behavior analysis tools, build a hypothesis aimed at increasing conversions. For example, “Increasing the size of the CTA button will make it more prominent and will increase conversions.”
Test your Hypothesis:
Create a variation per your hypothesis, and A/B test it against the original page. For example, “A/B test your original home page against a version that has a larger CTA button.” Calculate the test duration with respect to the number of your monthly visitors, current conversion rate, and the expected change in the conversion rate. (Try this free Bayesian A/B Testing Calculator)
Analyze Test Data and Draw Conclusions:
Analyze the A/B test results, and see which variation delivered the highest conversions. If there is a clear winner among the variations, go ahead with its implementation. If the test remains inconclusive, go back to step number three and rework your hypothesis.
Report results to all concerned:
Let others in Marketing, IT, and UI/UX know of the test results and the insights generated.
