Course Description
In this quick tutorial, we go over A/A testing. A/A testing is the tactic of using A/B testing to test two identical versions of a page against each other. Typically, this is done to check whether the tool used to run the experiment is statistically fair.
What You'll Learn
- Introduction to A/A testing
- How to use A/A testing to help conduct A/B or multivariate tests
Hi, and welcome to this quick introduction to A/A testing.
So we’ve just covered what is an A/B test and what is a multivariate test, but what is an A/A test? As mentioned in the previous video, if something is broken or the data is messy or maybe you don’t randomize correctly, this will have an impact on your experiment. You should make sure that your A/B test is being conducted properly first by setting up an A/A test.
An A/A test is when you compare the identical experience on a different random set of users. For example, in your original version, version A, our control is a green “Buy Now” button, and your alternate version, version B, your treatment, is also a green “Buy Now” button.
An A/A test is used to compare the identical experience and is used for validation of setup. It is good to set up an A/A test to show a random subset of users the same experiment and another random group of users the same experiment. You should have different users and different data and if it is the same experience but the metric is different something is wrong.
So, let’s say I’m running a drug test. Half of the people I am testing the drug on get one type of medication the other half can get the same type of medication. So, what kind of problems can there be if it is the same experience but the metric is different, the reaction is different, something is wrong. Again, once you set up your A/A test, you can then move on to conducting your A/B or multivariate test.
Thanks for watching. Give us a like, if you found this useful, or you can check out our other tutorials at online.datasciencedojo.com.
Blair Heckel - Blair holds a Bachelors degree in Marketing from Washington State University and has a background of leading data-driven marketing campaigns.
