The Need For A/B Testing Designs In Performance Marketing

Is there a time that you shouldn’t test a marketing campaign’s ad creative? This was a question raised by Amrita Mathur, VP of Marketing at Superside, at one of our Ad Designpalooza sessions. The panelists, Shanee Ben-Zur, CMO at Crunchbase, and Hakim Garuba, Performance Marketing Manager at Square, agreed that while testing is important, there are some cases when it simply doesn’t make sense.

Here are some quick testing rules to follow:

1. Sample Size Matters: You shouldn’t test the ad creative when the sample size is too small. Small sample sizes don't provide enough information to draw insights from.
2. No Budget, No Test: If there’s not enough budget to see through your experiment to acquire the right amount of data, then what’s the point?
3. Stakeholders Must Be Aligned: For creative a b split testing to be successful, stakeholders need to agree on their standards of success and what the main KPIs are.
4. Good Measurement is Key: You need to be able to accurately monitor and measure your data. Without good (clean) data, you can’t trust the results of your A/B test.
   

As performance marketers, A/B testing or split testing is not a new concept. It’s a marketing tactic that we often use to enable us to make data-driven decisions, especially in digital advertising. We research, analyze, hypothesize, test, launch and the cycle repeats itself. If executed properly, the results can have a positive impact on our bottom line.

There have probably been many times where you redesign a banner ad, revise a landing page design or make minor tweaks based on trends or personal preferences. The truth is this method doesn’t always work, especially if you’re looking to scale, test, and iterate complex designs. Being a performance marketer often means finding the optimal mix of offer, copy, and design. Therefore, if you're not split testing or landing page a b testing your designs, are you really optimizing for performance? To get you ready for split testing, this post will cover;

• Why A/B testing is important in Design
• The main benefits of A/B testing
• How to A/B Test design at scale
• A/B Testing Best Practices And Example

Why is A/B Testing Important in Design?

A/B testing is an important evaluation tool in the design process. The design process isn’t just an art; it’s also a science that relies on cold, hard data from the research and ideation stages. Determining which version of your landing page has the most impact, or which is the winning variation of your social ad is key in understanding the types of design elements to use to improve conversions or increase the number of leads.

3 reasons why every performance marketer needs to be testing variations of their designs:

1. Running A/B tests allows performance marketers to take the testing results and share that newfound information with their designers.
2. This user behavior data gets fed back into the design process to hypothesize, test, launch, and iterate, spurring new creative ideas and approaches.
3. And overall, A/B testing is an effective tactic to use especially if you involve your designer throughout the entire process. Better ROI, anyone?

TIP: Your designers shouldn’t be just task takers— the real magic happens when designers are given the chance to be strategic, creative thinkers. But they can’t do this without knowing how their designs are performing, and what the campaign goals are.

The 4 Main Benefits of A/B Testing

1. Improved Content Engagement

A/B testing forces you to evaluate every aspect of your creative. This means that while creating multiple variants, you’re also creating a list of potential improvements that may be statistically significant. Consequently, using an a/b testing tool makes the final version of the page or creative better for your target audience.

2. Increased Conversion Rates

A/B testing may be a longer process, but when executed properly, will help you improve clickthrough rate, convert more leads, and ultimately, increase sales. Better customer engagement means more customer conversions, which leads to increased sales volume for your business. Through testing designs and different call to action buttons, it is possible to improve conversions rates.

3. Ease of Analysis

With A/B testing, it’s simple to analyze real, factual results—no more guessing. The data helps you determine a “winner” and a “loser” based on straightforward metrics (i.e. time spent on page, conversions, etc.). There’s no ambiguity or room for swaying your opinion over the data. As you know, the most beautiful designs aren’t always the most effective, so A/B testing lets the data speak for itself!

4. Reduced Risks

A/B testing can help you examine website visitors and audience behavior on your creative before committing to major decisions and significant company budget.

We should also mention that there are a few downsides to over-testing your creative. Although testing designs can often lead to better outcomes in the future, there are times when it doesn’t make sense to test at all. If you’re keen to learn more, we’ve got a whole blog on how to balance your data and creativity! Use the data wisely—the data should work for you, you should not work for data.

How to A/B Test Design at Scale?

A/B testing a handful of variants is manageable, but have you ever tried to do it for dozens, if not hundreds, of variants? Let me put it into perspective for you:

As the Performance Marketing Manager at Superside, I recently launched a Facebook ad campaign to test which ad creative resonates with our target audience. We ended up with 36 design variants and two different text elements with a total of 72 ad variants. Thankfully for us, Superside has a scalable design team to help us execute this. This may not be the case for you.

Here’s what you can do to A/B test design at scale:

1. Improve Internal Processes

It’s important to involve your design team throughout the design process, which includes reviewing results from your tests. This helps your designers understand what you’re testing and why, so that they can constructively contribute with ideas on what to test next.

Advantages:

• Commitment: You receive an upfront commitment from your design team to contribute to the test by developing the iterations.
• Goal alignment: With your design team involved, you can ensure your standards for success for the test are aligned.
• Understand Return On Ad Spend (ROAS): A/B testing helps to evaluate and efficiently allocate your advertising dollars.

Disadvantages:

• Resource intensive: You may not have the humanpower or budget to execute a successful A/B test.
• Prior commitments: A/B testing can pull your design team away from higher priority tasks.

2. Review External Resources

If you don’t have the internal resources to scale design, this is an opportunity to work with agencies, freelancers, or a solution like Superside who can build creative iterations and tests for your campaign.

Advantages:

• Time to hire: Work with talented, quality designers without the entire hiring process.
• Faster delivery: For example, when you work with Superside, we offer 12-hour turnaround times.
• Cost-effective: Cost varies based on who you work with, whether it’s freelancers for one-off projects, or tapping into external design teams like Superside at ⅓ of typical agency costs.
• Reliability: Work with an agency or freelancer with a good track record of projects similar to yours
• Easier: The briefing process can be easier. Superside has a 10-minute brief.

Disadvantages:

• Limited capabilities: Depending on your design needs you may have to work with several parties. This isn’t the case at Superside – we have a full-stacked team with a range of skills sets to execute your design needs.

3. Build a Strong Tech Stack

Review and acquire tools that will make A/B testing design at scale easier for you and your design team.

• Google Optimize is a free website optimization tool developed by Google that helps marketers and webmasters increase visitor conversion rates and visitor satisfaction by A/B testing website content.
• Superside offers a design management platform that is free to all subscription customers. It lets you easily collaborate with design previews so you and your stakeholders can share and comment on design files, organize and share all your brand assets, and stay connected with your team on ‘Superchat’ to exchange files, files, and feedback.

Best Practices for A/B Testing Design

Now that we’ve covered most of the basics of A/B testing design, the last question we want to ask is how do you conduct efficient A/B tests for design? We spoke with Superside’s Amrita Mathur, VP of Marketing and Anneke King, Executive Creative Director, about their best practices when planning and carrying out A/B testing, and here’s what they shared with us:

Define What Creative Elements You Want to Test (+Metrics)

To start your A/B testing on the right foot, determine what variable(s) you want to test – copy, graphic, layout, etc. You can test one variable at a time or test more than one variable (known as a multivariate test) depending on the complexity of your A/B testing. It’s also important to have key performance indicators (KPIs) defined and baseline results to compare to (ex. current conversion rate).

If you haven’t already, loop in your designer or design team at this stage. As previously mentioned, Superside recently launched a Facebook ad campaign with 72 ad variants. When we knew we’d be A/B testing, we looped in Piotr Smietra, Marketing Creative Director, to get his input on what variables we should be testing and an idea of what the timeline would be to deliver those 72 ad variants.

Create a Strong Hypothesis

A solid, user-centered hypothesis is a must for your A/B test. A strong hypothesis can keep your evaluation focused on the relevant metrics rather than filling gaps with assumptions. Without a strong hypothesis, you can easily determine if an A/B test is even worthwhile to make and test those variables.

A few months ago, Superside conducted an A/B test on our top of funnel landing pages. This was our hypothesis: “If we change our landing page background to a more lively/colored background, then we’ll see an increase in conversions, because the webpage is more appealing to our audience.” And the results? Our hypothesis was true, so from now on you’ll be seeing more lively landing page backgrounds from us!

Failed Tests are a Good Thing

In an ideal scenario, your A/B test has a “eureka!” moment where there is a clear winner and loser. Sometimes, A/B tests fail to produce significant results. If that happens, run your experiment longer. If your hypothesis is disproven, don’t be discouraged. Even a failed test can share insights and opportunities for your next iteration, only increasing your chances of succeeding.

Create a Balance Between Design and UX

When you focus too much on “data-driven” design, user experience can be negatively impacted. A/B testing tends to focus on the quantitative data, but it’s also important to set qualitative KPIs (ex. registration, form submissions, etc.) when running an A/B test. A high-traffic landing page may sound great, but if the page isn’t receiving form submissions, then something needs to be changed to positively influence the user.

Give Experiments Sufficient Time

Conducting an A/B test in a shorter amount of time won’t get you your results any faster. That’s why it’s important to give sufficient time to conduct a thorough A/B test with enough participants or iterations to achieve strong, concrete results. As the saying goes, “slow and steady wins the race!”

Conduct A/B Testing Regularly

Things change over time and that applies to A/B testing. Make A/B testing a regular part of your process to ensure that you have the most up-to-date data on your design variables. A high-performing CTA button won’t always be “high-performing” and that’s when you’ll need to A/B test to see if there’s a better way to do it.

Every marketing team has a method to A/B testing. At Superside, we believe in the process of iterating and optimizing to understand our customer base and market at large. From ABM to content, our marketing team A/B tests design regularly. It also helps that we have a design team and creative support to help us execute (yes, we use our own solution as well!).

Start A/B Testing Design Today

Take this as your sign to start A/B testing design, whether that’s landing page designs or digital ad designs. The results of your creative tests provide valuable data that gets fed back into the design process to iterate over and over again until you have the winning variant.

For performance marketers who want to A/B test design at scale, these are the three main points to remember:

1. Involve your designers throughout the creative process.
2. Work with agencies, freelancers, or an advertising design service like Superside to help with the design workload.
3. Invest in the right tools to make A/B testing easier and more efficient for your team.

If you have started A/B and multivariate testing design, keep going because you’re on the right track!