How “Statistical Significance” Can Help You Optimize Your AdWords

Last modified on 5th January, 2007

I hope you won’t find my first post for the New Year boring because I am going to delve into the “exciting” world of statistics and probability 🙂

Don’t worry; I am not about to start a math discussion here, actually I am not a fan of complicated statistics theories either.

But I’d just like to discuss about two simple and free online tools that utilize the “statistical significance” concept to help you save time and improve your AdWords ads performance easily.

OK, so what does “statistical significance” really mean?

When a statistic is significant, it simply means that you are very sure that the statistics is reliable. You can find the full definition from Wikipedia here.

When you run a marketing split test, you have to make sure you run enough trials to be sure that the “winner” didn’t happen by fluke.

If you do not know how to create multiple ads for an Ad Group for split testing purpose, check out the AdWords Help Center here for an explanation.

So how does it apply to you? Let’s say in one of your AdWords Ad Group, you have got two ads variation. One gets a 1.3% CTR with three clicks total and the other gets a 2.3% CTR with four clicks total.

How sure can you be that better one i.e, 2.3% really better and will perform differently in the long run?

To answer this question, visit http://www.splittester.com and plug in your numbers and click “Calculate”.


A result will come up saying:

You are not very confident that the ads will have different long term response rates.

That means you can’t delete the low CTR ad yet. This is because the chance for the better ad to continue to generate a higher CTR in the future isn’t significant. In other words, you are still not sure whether both ads will perform differently in the long run.

Now, let’s say you’ve got 30 clicks for the first ad and 45 for the other, not three and four.


When you click “Calculate”, the result will say:

You are approximately 99% confident that the ads will have different long term response rates.

With that kind of result, I’ll buy a lottery in a blink of an eye. But that’s besides my point.

99% confidence means if you ran this test 100 times and got these results, the results would lead you in the right direction 99 times.

So now you can confidently delete the low performing ad and create a new ad to beat the CTR of the better performing one.

This sounds nice and all but wouldn’t it be nicer if we can do this automatically instead of manually entering data? Imagine if you manage a lot of Ad Groups. Optimizing them all can be tedious and time consuming as each ad requires a lot of individualized attention.

Here comes AdWords Optimizer from ClickMuse to the rescue. It works by looking at your AdWords data of all your text ads in a given Ad Group and then makes recommendation which ads to keep or dump.


If it finds two ads have similar performance, it’ll suggest you keep one and create a new one. It also calculates your potential CTR Gain if you dump the losing ad.

I have been using AdWords Optimizer for over a month now and find it to be a time saver. But a question cropped up in my mind while using this tool.

While striving to improve CTR is always good, there’s another issue that you have to consider when using this tool. The tool focus on getting more clicks but fall short when it comes to identifying ads that could help you get higher conversion rates.

At the moment this seems to be its major limitation.

Instead of writing another page or two here, I think you should check out a related article by MindValleyLabs at their Internet Marketing Blog.

The blog post answers the question whether it is better to optimize your click through rate (CTR) or your conversion rate (ROI)

Full article is available here: Google AdWords Tip: Optimize for Leads NOT CTR and NOT ROI


An engineer by training, Victor has been working full-time online as an Internet marketer, a programmer and an app developer since 2001. He has been blogging at Sabahan.com since 2006 sharing his experience and teaching people how to make money online. Click here to join his private Facebook Group for bloggers.

  • […] Another post you can check How “Statistical Significance” Can Help You Optimize Your AdWords […]

  • […] out this related article I wrote earlier about identifying successful ads using Statistical […]

  • >

    Sign Up Below... For Tips and Tools to Help You Build a Better Blog

    >> Sign Up Now and get access to:

    • chevron-circle-right
      The exact methods I use to make money
    • chevron-circle-right
      Resources to increase traffic to your blog
    • chevron-circle-right
      The latest updates from Sabahan.com

    We hate spam. Unsubscribe anytime.