What is AB testing - Explained.
- AB testing is putting two experiences against each other to see which one is better.
- AB testing is the best way to make data-driven decisions.
- You need to know about AB testing because it can help you increase the value or add to the value.
- The reason why AB testing can do this is due to it allows you to quantify the changes and know which changes are influencing the bottom-line metrics for your business, so you know what’s driving the revenue.
- AB testing happens in your digital spaces,
- Websites
- Apps
- Social media channels your
- Emails your
- Paid advertising
- Think of it like digital experimentation each test helps you learn which variation is better than the others.
- In an app test, the original page or experience is the control, or the A experience and the challenger is the B experience which is the new version that you’ve made to compete against the control.
- These two versions are competing to see which one does better for your visitors.
- Every champion was once a contender who refused to give up, you’re not about a hard hit it’s about being the winning test variation to keep moving forward
- How do you know which of these variations is the winning experience even though both look extremely good only one of them can be the winning experience.
- The best experience is determined by a couple of things the first
- Your visitors – they are split and randomly assigned to the a or to the b experience. This randomization helps you avoid sampling bias.
- Another benefit of a b testing is these visitors don’t know that they’re in the test, they don’t know they’re being tested, and so the result is their unbiased behavior. They vote with their behavior if they don’t like the experience then that experience doesn’t win. This absence of bias gives you good data.
- Your visitors are on a journey and the path they take will hopefully lead them to their goal.
- You want them where x marks the spot you want them where the treasure is.
- Not all traitors silver and gold mate discovering the winning a b test variation.
- Now that’s priceless and your primary metric is the treasure that you want your visitors get to.
- If you’re a retail company and you’re selling things on your site then your primary metric would be orders or revenue per visitor.
- If you’re trying to get leads, then your primary metric would be signups.
- For many sites the primary metric is content consumption or time on site.
- If they’re trying to display ads so that your ultimate goal is to get your visitors to do more of the thing you want them to do, whatever your primary metric is, whatever the goal of your site is, you need to determine that before the test is run.
- Then once you have enough data you can calculate the lift and the confidence to see which of the variations did better than the others.
- Your lift and confidence numbers will tell you how much better or worse your new variation is and how much you can rely upon that data.
- Based on the statistical confidence this brings us back to how a b testing can help you.
- Personally, if you’re looking for that raise or promotion then you can show how the tests you’re running have been able to increase your primary metric and increase the revenue for the business.
- So why don’t businesses just help their visitors get to the treasury, of course, they want to but they don’t always know what their visitors want.
- In fact, the visitors don’t know what they want either.
- Most of the time in the words of captain jack sparrow the problem is not a problem. The problem is not a b testing to solve the problem.
- AB testing gives the business insight into what leads the visitors down the perfect path to get to the treasure.
- The journey for your visitors is optimized as you experiment and test everything in the digital experience.
- You can test the flow between departments and categories and pages.
- You can test layouts of pages and elements.
- On the page, you can add or remove existing elements to find out if they belong.
- If they’re helping the visitors along the path or if they’re hurting the visitors on the path.
- You can test images, you can test heroes, you can test copy, you can test calls to action, and you can even test back-end algorithms, that are recommending products or content for your visitors.
- Each tested change that you make will allow you to create this personalized roadmap for your visitors so they’re better able to go along that path and complete their journey.
- Increase revenue & conversions
- Rapid iteration
- Learn what works
- Uses actual site visitors
- data-driven decisions making
Where to AB Test
- All website experiences
- Paid advertising
- Mobile apps
- Marketing campaigns
- Emails
- Site pages, Flow & Elements
- Business model
- Backend functionality & Algorithms
- New Products and services
How to get started
- Get a testing tool
- Define the primary success metrics
- Define the idea/question to be tested
- Define variations and learning
- Create the variations
- Measure and activate
- Analyze the results
- Document learning and evangelize
- Ideation
- Repeat