Mihir works as a marketing manager. With limited budgets, he has used a variety of techniques to increase revenue for his company. To make the most of every campaign and maximise his marketing objectives, he relies on A/B Testing. Why? 

  1. Because A/B Testing is one of the simplest and cost-effective ways to evaluate the traffic and audience to his website
  2. He can monitor the performance of his website using statistical analysis.
  3. Based on the website visitor behaviour, he can not only optimise his website, but at the same time, he can improve the site’s performance and generate more revenue. 

Some other key benefits of A/B Testing include: 

  1. Save money by identifying processes that offer better returns
  2. Increase profits by improving conversions
  3. Reach a bigger set of audience
  4. Improve bounce rates 
  5. Get better ROI from existing traffic
  6. Enhance website content
  7. Solve visitor pain points.

Where can he run A/B tests? 

With A/B Testing, Mihir could practically test anything and everything: from a simple website headline to a critical, lead-generation call-to-action button and even the placement of various page elements. Not only that, Mihir could run A/B tests on different websites, Mobile apps, Paid Advertising, Marketing Campaigns and emails.

A/B Testing: Mistakes to Avoid

To get seamless and error-free results, here’s a list of some of the most common mistakes that you must avoid when you are running an A/B test:

  1. Testing with a small sample size

Good A/B Testing requires around 25,000 visitors to reach a significant sample, according to a VentureBeat article. Hence, you must not conduct A/B Testing with a smaller sample size, as this would result in an incorrect representation of the total population and irrelevant findings.  

  1. Little Retesting Options

A/B Testing is an iterative process. It is recommended that you do it more than once and that too once in a few months. However, in some cases, it is seen that companies give up on A/B Testing after their first test and do not opt for retesting. Therefore, instead of stopping after a successful test, keep on testing each element repetitively to produce the most optimised version of it even if the campaign is successful.

  1. Complicating with too Many Metrics

Although complex tests may look useful, they may not always be efficient. Looking at too many metrics in your software or testing too many elements of a website together can result in “spurious correlations” and ineffective results. Rather, focus on relevant metrics that matter to avoid random fluctuations in the results and inferences.

  1. Not Completing the tests.

Most marketers believe that running a test for a long duration or a very short period can produce insignificant results. Also called p-hacking, this is a form of bias due to ‘selective reporting’ and can produce inaccurate results. One of the common mistakes that you can avoid is to let each test run its course, even when you see the results in real-time. Effective A/B tests can take just a few days to give the correct results, while others can take many weeks.

However, on a general note, it’s always a good idea to give your A/B test around two weeks to run before going forward with any strategic decisions.

Companies that have used A/B Testing in their strategy

Google, Booking.com, Netflix and Amazon are a few names that have delivered excellent user experience through continuous and structured A/B Testing as part of its marketing strategies. 

For, e.g., before deployment, every change on Amazon’s website is first tested on their audience. Apart from this, each page is optimised, using user insights and website data and every step leading to the CTA is simplified to improve the overall user experience significantly.


For Mihir, A/B testing proved extremely valuable for improving his website’s conversion rates by eliminating all weak links and creating the most optimised version. 

Similarly, when done correctly, A/B testing can help you significantly improve results for your website.