What is AB testing, Benefits, Where, What & How?

Table of Contents


What is AB testing - Explained.

  • AB testing is putting two experiences against each other to see which one is better.
  • AB testing is the best way to make data-driven decisions.
  • You need to know about AB testing because it can help you increase the value or add to the value.
  • The reason why AB testing can do this is due to it allows you to quantify the changes and know which changes are influencing the bottom-line metrics for your business, so you know what’s driving the revenue. 
  • AB testing happens in your digital spaces,
    • Websites
    • Apps
    • Social media channels your
    • Emails your
    • Paid advertising
  • Think of it like digital experimentation each test helps you learn which variation is better than the others.
  • In an app test, the original page or experience is the control, or the A experience and the challenger is the B experience which is the new version that you’ve made to compete against the control.
  • These two versions are competing to see which one does better for your visitors.
  • Every champion was once a contender who refused to give up, you’re not about a hard hit it’s about being the winning test variation to keep moving forward
  • How do you know which of these variations is the winning experience even though both look extremely good only one of them can be the winning experience.
  • The best experience is determined by a couple of things the first
    • Your visitors – they are split and randomly assigned to the a or to the b experience. This randomization helps you avoid sampling bias.
    • Another benefit of a b testing is these visitors don’t know that they’re in the test, they don’t know they’re being tested, and so the result is their unbiased behavior. They vote with their behavior if they don’t like the experience then that experience doesn’t win. This absence of bias gives you good data.
  • Your visitors are on a journey and the path they take will hopefully lead them to their goal.
  • You want them where x marks the spot you want them where the treasure is.
  • Not all traitors silver and gold mate discovering the winning a b test variation.
  • Now that’s priceless and your primary metric is the treasure that you want your visitors get to.
  •  If you’re a retail company and you’re selling things on your site then your primary metric would be orders or revenue per visitor.
  • If you’re trying to get leads, then your primary metric would be signups.
  • For many sites the primary metric is content consumption or time on site.
  • If they’re trying to display ads so that your ultimate goal is to get your visitors to do more of the thing you want them to do, whatever your primary metric is, whatever the goal of your site is, you need to determine that before the test is run.
  • Then once you have enough data you can calculate the lift and the confidence to see which of the variations did better than the others.
  • Your lift and confidence numbers will tell you how much better or worse your new variation is and how much you can rely upon that data.
  • Based on the statistical confidence this brings us back to how a b testing can help you.
  • Personally, if you’re looking for that raise or promotion then you can show how the tests you’re running have been able to increase your primary metric and increase the revenue for the business.
  • So why don’t businesses just help their visitors get to the treasury, of course, they want to but they don’t always know what their visitors want.
  • In fact, the visitors don’t know what they want either.
  • Most of the time in the words of captain jack sparrow the problem is not a problem. The problem is not a b testing to solve the problem.
  • AB testing gives the business insight into what leads the visitors down the perfect path to get to the treasure.
  • The journey for your visitors is optimized as you experiment and test everything in the digital experience.
  • You can test the flow between departments and categories and pages.
  • You can test layouts of pages and elements.
  • On the page, you can add or remove existing elements to find out if they belong.
  • If they’re helping the visitors along the path or if they’re hurting the visitors on the path.
  • You can test images, you can test heroes, you can test copy, you can test calls to action, and you can even test back-end algorithms, that are recommending products or content for your visitors.
  • Each tested change that you make will allow you to create this personalized roadmap for your visitors so they’re better able to go along that path and complete their journey.
  • Increase revenue & conversions
  • Rapid iteration
  • Learn what works 
  • Uses actual site visitors
  • data-driven decisions making

Where to AB Test

  • All website experiences
  • Paid advertising
  • Mobile apps
  • Marketing campaigns
  • Emails
  • Site pages, Flow & Elements
  • Business model
  • Backend functionality & Algorithms
  • New Products and services
What is AB testing

How to get started

  • Get a testing tool
  • Define the primary success metrics
  • Define the idea/question to be tested
  • Define variations and learning
  • Create the variations
  • Measure and activate
  • Analyze the results
  • Document learning and evangelize
  • Ideation
  • Repeat

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

SEM | Search Engine Marketing Blogs
Zabi Niazi

3 Strategies Help Seasonality Planning

Seasonality Planning Strategies During seasonal times, it’s important to focus on campaign efficiency while also connecting with your potential customers.  Three strategies help Seasonality Planning:

Read More »
SEM | Search Engine Marketing Blogs
Zabi Niazi

Actionable Strategies – Increase Backlinks

Backlinks also known as one-way links or Inbound Links are links from one website to a page on another website. Google and other major search engines consider backlinks “votes” for a specific page.

One strategy to increase the chances of showing Google page 1 or higher rankings is to increase the number of backlinks from credible and authoritative sites.

Read More »
SEO | Search Engine Optimization Blogs
Zabi Niazi

Structured Data and Schema Markup

In this blog, I’ll share what structured data is, how it’s useful for both Google and you, how to get started using it, and some best practices if you already do.

Read More »
On page SEO Checklist
SEM | Search Engine Marketing Blogs
Zabi Niazi


What is URL  URL stands for Uniform Resource Locator. The location of a webpage or file on the internet. The URL for the website you

Read More »