Search Engine War Blog : « Google Testing Yahoo Like Quick Links | 'The Guava Five' - Rednose Day 2009 »

Visible Quality Score has Nothing to do with Relevancy - at least initially

Thursday, 12 March 2009


Untitled

I started this investigation thanks to an article over at PPC Hero in which an AdWords representative stated that “The moment you load a keyword into AdWords, it receives a Quality Score. This initial Quality Score is determined by each keyword’s performance history for other advertisers.”
Later PPCPROZ also published a follow up email from the AdWords team stating that “both factors [the historic performance of the keyword for other advertisers and the specific quality of the account structure and landing page] are weighed initially at account creation, and that performance data will be taken into account as soon as the ad starts to serve.”

I believed the weighting that the AdWords reps seemed to be putting on this historic performance factor was too high. In my comment on the blog I made an assumption that relevancy, both between the keyword and the ad and the keyword and the landing page, would be the number one most important factor in determining a keywords initial quality score. I did this because the effect relevancy can have on a keyword’s CPC is quite dramatic. Competitor data would, I thought, be used be used as a metric to give the ads an “average” QS within the landscape. I decided to run an experiment to investigate initial Quality Score (IQS) and how it was affected by relevancy. It proved my assumptions to be very wrong indeed!

Methodology – the casual observer and the easily bored should feel free to read just the text in bold and my final observations.

Notes: Where I refer to Quality Score I am of course talking about the AdWords interface’s Visible Quality Score.
Multiple match types were added of each keyword, but these made no difference – the visible QS was the same for a keyword no matter its match type. I have removed the multiple match type entries for ease of reading.
All keywords were set up with a max CPC of £3.00, this factor should be irrelevant to the QS.

In order to test initial Quality Scores I set up a new campaign in a new account (so that there would be no issues with residual QS from other campaigns or adgroups). To begin with I set up 4 ad groups, each with one ad, the destination page of each ad being www.guava.co.uk.

The first ad was highly targeted to both the keywords I was using and the landing page. Almost every word in the ad appears on the landing page, and I even included a phrase from the ad directly in the keyword list.

Ad A: Keyword set 1:
Guava UK Services guava
We search online to find our expert guava uk
clients a fresh Guava in the UK ”fresh guava in theuk”
www.guava.co.uk/guava guava search
guava clients

All of the keywords came back with visible quality scores “OK”.

The second adgroup used exactly the same ad but this time had keywords focussed at the landing page content and deliberately not any of the copy in the ad. I actually took keywords suggested to me by the AdWords interface as being relevant to my landing page in the “sample keywords based on a scan of your site” section.

Keyword Set 2:
ppc
pay per click
pay per click advertising
seo
search engine optimisation
search engine optimization
web design

Whilst a majority of these keywords came back “ok”, the SEO keywords actually had a “poor” visible quality score. Landing page content and its relevance to your keywords obviously has very little if any affect on IQS.

The remaining two adgroups both had the same ad, which bore no relationship to the landing page content. One adgroup had keywords relevant to the ad, but obviously not the landing page, and one with keywords relevant to neither the ad nor the landing page (nor each other).

Ad B:
Adopt a Penguin
Will you care for a baby penguin?
Literally pick up a penguin today!
www.guava.co.uk/penguin

Keyword Set 3: Keyword Set 4:
Penguin loft insulation
baby penguin tennis racket
penguins kite flying
adopt a penguin trips to mars
norwegian translation
stop snoring

The keyword ‘adopt a penguin’ received a “great” quality score. Was this because of ad relevancy?
No – the keywords ‘loft insulation’, ‘trips to mars’ and ‘stop snoring’ also received a “great” quality score. This is interesting because according to Google a “great” QS means “The keyword is very relevant and may have a high click-through rate (CTR), relevant ad text and a unique, relevant landing page.” Well, they don’t have relevant ad text or landing page content, so they must have high “historic average” CTRs.
In fact these weren’t all random keywords, I know that ‘loft insulation’ and ‘stop snoring’ are both high competition terms, with good CTRs, so we must assume that it’s this that makes the difference.

It seems relevancy of the landing page and ad copy to the keyword has no real effect on initial Visible Quality Score.
Does this mean ALL the IQS is a result of historic keyword performance data? I couldn’t turn down the opportunity to see what happened with some more ‘historically’ top performing keywords. I selected half a dozen high volume, high demand (and so presumably high CTR) keywords and set them up with relevant and irrelevant ads as below.

Keyword Set 5:
cheap flights
spread betting
credit cards
mobile phone
car insurance
used car

Ad C: Ad D:
Cutlery Warehouse Spread Betting Insurance
Save on your fork and spoon bills Spread the cost of car insurance.
with a spork. Its scoopy & pointy! Mobile phone insurance with credit
www.guava.co.uk/sporks www.guava.co.uk/cheap_credit

The keyword-rich nonsense in Ad D had no effect. In both ad groups the QS for almost every keyword was the same - “OK” for spread betting, “Great” for everything else.
The exception was that ‘car insurance’ had an “OK” QS in Ad D’s group, but a “Great” QS in Ad C’s group. In the same campaign the keyword ‘car insurance’ has a worse QS when associated with an ad containing the term “Car Insurance” than it does when associated with one about cutlery!

Relevancy does still have one effect at this early stage of a campaign, that of CPC.
In our irrelevant ad campaign Spread Betting had a min first page CPC of £6.50 and Car Insurance a min first page bid of £5.75. In the ad group with the more relevant ad these were both £0.50 cheaper.
CPC should be tied up with QS. Back in the days of minimum bid we were told that “minimum bid is a direct reflection of Quality Score” to the extent that your visible quality score rating could be extrapolated by looking at your minimum bid. Obviously this is not the case anymore.

Final Observations

The difference in CPC demonstrates that AdWords does look at your ads and take the relevancy of keywords and ad copy into account. However it doesn’t seem to have any effect on the initial visible quality score attributed at keyword level. It seems this is almost entirely down to the historic CTR of that keyword. This is a big lesson for me. In a situation where I had uploaded a keyword and it was given a “poor” IQS I might well have looked at the ad and landing page to tried and improve this. This isn’t the right tactic at all – indeed it’s a waste of time – you might find you’re better off getting out there with an aggressive bid and generating a good CTR to wipe the historic QS and hopefully keep your bids lower in the long-term.

It is evident the affect of relevancy on Quality Score is not reflected at all through the visible Quality Score column at the point of keyword upload.

Comments

gaz-j

do you think that over time most keywords will be being optimised for and so the marketplace is getting tougher all of the time? eventually is there going to be a major change in googles algorithm?

kelvin newman

"This is a big lesson for me. In a situation where I had uploaded a keyword and it was given a “poor” IQS I might well have looked at the ad and landing page to tried and improve this. This isn’t the right tactic at all – indeed it’s a waste of time – you might find you’re better off getting out there with an aggressive bid and generating a good CTR to wipe the historic QS and hopefully keep your bids lower in the long-term."

Great piece of advice there....

Well worth taking the time to do the experiment

Matt Whelan

@gaz-i In terms of PPC and QS, it would be self defeating to update the QS algorithm just because people's ads are becoming more relevant - partly because it's there to ensure relevant ads show in the bid landscape, and partly because the better EVERYONE's ads get, the higher CPCs, and Google's revenues, become!

@Kelvin thanks!

Tom

interesting article & findings!. Certainly has the potential to change the way you work with new keywords to an account on Google and your optimisation.

Search Engine Optimization Company

I would agree with the relevancy out weighing the historical data.

Pat

If initial quality score has nothing to do with the LP, why is the ad "rarely shown" due to the quality score? They can't rarely show it without looking at how relevant the ad is, can that? That makes no sense.

cheap flights

I too agree that it is necessary to get travel advice from a trustworthy source. This will definitely be of great help to anyone who wants a joyful journey.

Francis

Hi, I must admit that this blog on is really nice. Though the para 1 is introductory, the most beautiful thing I liked about it is the conclusion part. You have concluded in an unbiased way presenting the facts about .

Buy Dissertation

I am really impressed by this blog! Very clear explanation of issues is given and it is open to everyone. thanks for posting this work of yours..this is very good!

The comments to this entry are closed.

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451c37d69e201127963fa6728a4

Listed below are links to weblogs that reference Visible Quality Score has Nothing to do with Relevancy - at least initially:

Subscribe to this blog's feed

Add to My Yahoo!
Subscribe with Bloglines
Add to Google
Subscribe in NewsGator Online

Add to My AOL
Add to Technorati Favorites!
Add to netvibes