If you’ve ever tried to get approval before launching a marketing asset, you’re familiar with the purgatory many marketing decisions fall into.
Don’t you think this landing page is too long?
Should we add more than one CTA button?
Should we include a name in the email subject line?
There are dozens of decisions to be made for each marketing asset that’s created. Small decisions can have a big impact on whether the asset converts.
How do you decide what to do? Go with your gut? Do whatever the CEO says without any pushback? Use your best guess based on some quick wins in the past?
Allow me to introduce you to A/B testing.
A/B testing in a nutshell
What is A/B testing?
In a nutshell, A/B testing (also called split testing) is creating two different versions of a marketing asset (landing pages, subject lines, CTA buttons, exit pop-ups, click-through rate, etc.) and exposing it to two different audience segments.
After a specified amount of time, you’ll analyze the test results of the different segments to assess which were most successful in increasing your chosen business metric.
A/B testing is helpful in a variety of ways:
It gives you data for the “why” of your content strategy: If your manager asks why you use short landing pages with three red CTA buttons, you’ll have a proven data-based answer.
It increases your conversion rates: You’ll increase sales by doing what works instead of wasting resources on what doesn’t. Over the course of many A/B tests, your marketing funnel will operate like a well-oiled machine with data-driven solutions being deployed in the creation of all marketing assets.
It gives you useful data: Even if both your A and B tests fail at producing the desired result, you have examples of what didn’t work and can focus your efforts on something else. So there’s really no such thing as failure.
It helps optimize existing website traffic: If you are in the frustrating position of generating lots of traffic but only a small percentage is converting, A/B testing will allow you to solve the mystery and turn traffic into sales.
It gives user insight: Everything you learn from your A/B tests can be used in your strategy moving forward to hone in on your target audience. From a website re-launch to new email campaigns, you’ll understand what your target audience wants.
What elements can you test?
You can run A/B tests on anything you’d like, provided you have some variations to experiment with and a clear hypothesis.
Here are some of the most common areas that benefit from A/B tests:
Websites
- Homepage images
- Navigation bars
- Video vs. text
- CTA buttons
- Page layout
- Pop-ups
- Colors
- Fonts
Landing pages
- Short-form vs long-form copy
- Contact form fields
- Checkout page
- Writing styles
- Pricing plans
- Social proof
- Headlines
Email campaigns
- Number of emails in the campaign
- Social share buttons
- Number of images
- Length of text
- Video vs. text
- Subject lines
- Email tone
How to design an A/B test
There are a few things you’re going to want to be certain to establish when designing your A/B test.
Establish your A/B testing goals: What business objectives are you trying to achieve? Data for data’s sake is cool and all, but you’re trying to drive growth. Are you trying to increase website traffic? Convert traffic? Decrease bounce rate? Gain subscribers? Figure out what your objective is first.
Determine a hypothesis to test: This step is crucial. Based on what you know of current user behavior, what’s your hypothesis about how to reach your goal? Clearly define your hypothesis and why you believe your proposed change(s) will make a difference.
Create test variations for your hypothesis: Next, select your variations. This can be the colors of a button, the placements of a layout, the length of a headline, or similar. Choose a variation on the testing elements you are designing your experiment around.
Establish a time frame for your test: Determine how long you want the test to run. It should be long enough to gather a sample size of statistical significance. The general rule of thumb is to run for at least two weeks but no longer than eight weeks.
Analyze your results: Go through your results with a fine-tuned comb. Of course, you’ll be looking for statistically significant differences, but don’t discount practical implications, especially if some of the segments don’t behave in the way that you expect. Use your insights to refine the dimensions of future tests and to determine which variations should be adopted for long-term growth.
Document your findings: Whatever the results of your test, write them down. Data lost is data that might as well never have been gathered. You don’t want all your efforts to be lost in time or left unprovable when you are making a case for a change to be made.
Adjust your marketing campaigns accordingly: Everything you discover in your A/B testing won’t do you any good if the data isn’t integrated into your marketing plan. Use the data you collect as a blueprint to adjust your current marketing strategy and a foundation for designing those in the future.
Retest: Don’t be afraid to test and test again! Something that was true for certain segments at a certain time may not be true a year later. COVID-19, for instance, skewed many marketing metrics that many considered set in stone. Your audience will change with time, and your approach will also need to get the most reliable data.
Get outside help: Sometimes, you may need to consider outside help when doing all of the designing and statistical analysis of A/B testing. Hiring freelancers or agencies can be a big help with all the elements of your marketing strategy that require many A/B tests, such as email campaigns.
A full-service email marketing agency offers expertise in designing effective tests, analyzing results for actionable insights, and continuously refining email campaigns for maximum impact. A collaboration like this can take one element off your plate as you focus on your website’s conversion rate.
A/B test examples
Let’s look at some variables that real companies have A/B tested.
A/B test example #1 – homepage
The clothing brand StudioSuits specializes in men’s formal wear including shirts, pants, and men’s linen suits.
They wisely used A/B testing during the holiday sales rush.
Before the holiday season was in full swing, their homepage prominently featured a heading focused on custom suits for men.
However, amid holiday fervor, StudioSuits employed A/B testing, experimenting with alternative headings like “Limited Deals” and “Special Offers” to infuse a sense of urgency.
This iterative testing helped them identify the most compelling content and ensured that the online store was optimized to resonate with customer preferences, ultimately bolstering conversion rates.
A/B test example #2 – call-to-action buttons
In three different case studies conducted by B2B companies (Hubspot, Dmix, and VWO), an A/B test was conducted using different colors for CTA buttons.
Across the board, the red button performed the best, with the margin of difference being as high as 34%.
While using the data of previous companies can be a good starting point, it’s important to test for your own brand as well.
For instance, if your brand is Coca-Cola, a red CTA button isn’t going to stand out. It’ll get completely lost in the overpowering presence of red in the rest of your branding.
What works well for one company or industry may not work the best for you. So don’t fall into the trap of taking too many shortcuts at the expense of doing your own research and collecting your own data. Keep evaluating the effectiveness of your strategies and refine your data collection methods.
A/B test example #3 – email subject lines
Email subject lines are a popular choice for A/B testing, and for good reason.
Marketers can expect a $36 return for every $1 spent on email marketing campaigns.
The Catholic jewelry e-commerce shop House of Joppa used an A/B test for their holiday email marketing campaign.
Screenshot provided by the author
You’ll notice in the A test email subject line sent November 27, House of Joppa highlighted “Cyber Monday” to entice the reader.
In the B test, they used “Flash Sale” and “Christmas Collection.”
Both email subject lines are meant to communicate the same message—they are having a 15% off sale sitewide—but this test allows them to see which subject line will have the highest open rate.
Common mistakes to avoid while A/B testing
No matter how carefully your A/B test is designed, falling into some common pitfalls can set you back significantly.
Here are some to avoid:
Being lax about documentation
Sitting down and typing out your current analytics, the set-up of your A/B test design, hypothesis, and results sounds about as exciting as watching paint dry. However, it’s pivotal for success.
Unless you write things down, you’re never going to remember the granular details of everything you’re testing or the current state of all your marketing assets.
Furthermore, you won’t be able to continue building on your success if you don’t track where you came from and where you’re going.
Lastly, you may not always be on your team doing the job you’re doing now. Good documentation guarantees that all the time and resources that went into gathering data aren’t wasted every time there’s a transition on the team.
Testing too many variables
It makes sense to want to gather as much data as you can with a single test. But if you are trying to determine which element performs better, you need to focus on that element.
Imagine, for example, you’re A/B testing a pop-up box on your home page prompting readers to sign up for your email list.
You want to increase the number of people who click your CTA button that currently says “Join Mailing List.”
The current text—“Join Mailing List”—is your control page, your A test.
However, you’re going to create a separate pop-up with a button that says “Add Me” to test which gets the highest click-through rate.
But you’d really like to know if a blue button or orange button performs better, so you set up your experiment to test that variable, too.
Well, now you have a problem.
If you get an increased number of subscribers with either test, you won’t know if the color or text was the determining factor.
If you want to test both text and color, you’ll need to do two separate A/B tests.
(There is a way to simultaneously test multiple variables at once. It’s called a multivariate test. But unsurprisingly, multivariate testing is more complex to execute and requires the expertise of advanced marketers.)
While it’ll take time and patience, once you know exactly the variables that convert the best, you can use your knowledge to optimize all of your marketing assets in the future.
Not using the right tools
The good news is there are many tools available to help you design and run A/B tests. With the right tool, you won’t need in-depth technical coding knowledge, even to run tests on more minute aspects of your web pages, such as authentication methods using SAML or OAuth.
The bad news is the wrong tools can sometimes hurt more than they help.
Some tools come with many functionalities you don’t need and will never use while being light on their A/B testing capacity. Others may require advanced knowledge of how to set up and run campaigns you don’t have and don’t have the time to learn.
You want a straightforward tool that’s easy to use and does exactly what you want it to do without extra hassle or expense.
Google Optimize (free and integrated with Google Analytics), Freshmarketer, and Crazy Egg are great places to experiment with A/B testing tools and learn what’s best for you and your company.
If you’re looking to expand your audience and are willing to invest in a service, consider exploring Google Ads alongside your email campaigns. It is essential to monitor Adwords competitors for optimal results.
Although analytical data can provide information on user behavior, it doesn’t explain the underlying reasons.
To gain insight into the causes behind these patterns, it’s important to understand what users are viewing and doing. That’s where a website session replay tool comes in handy.
You can finally learn exactly how visitors interact with your content, all without doing any heavy lifting. These insights are a gold mine for perfecting your future content for conversions.
Wrap up
A/B testing takes out the guesswork from the dozens of marketing decisions that arise when launching new marketing campaigns.
With a carefully designed A/B test, you can gain insight into how your users engage with your site and content and how to capture website traffic and convert it into sales.
While A/B testing can become as complex as you’d like to make it, many tools are available to help you design a straightforward and easy-to-analyze A/B test to help you get the most out of your marketing efforts.
If you want to ensure you’re getting the most out of your content, contact us about how we can help optimize your content library to be the most impactful.
