Has Google Changed Their CPC Formula?
Posted by Richard Stokes on March 17, 2011 to
In October, Google quietly rolled out changes in the manner in which they report keyword traffic estimates. What most advertisers don't know, however, is that there appears to have been a shift in the way CPC prices are being calculated as well.
What Changed?
As I wrote about in chapter 17 of The Ultimate Guide to Pay-Per-Click Advertising, the Google CPC formula has changed a number of times throughout the years. I examined CPC curves across a variety of keywords in different situations (high and low traffic, high and low CPC, high and low quality scores) and gave some general guidelines for what a PPC marketer could expect to encounter in real-world scenarios.
Back then, we saw that CPC prices typically started low near the bottom of the right rail and gradually increased with higher positions. We also discovered that Google was charging a premium of around 49% to "gladiator bidders" - those advertisers who were willing to pay high CPCs but did not have the quality score to break through to the premium positions. A typical CPC curve looked something like the following:
In late 2010, we discovered that the CPC formula appeared to have changed. We started seeing higher average CPC prices for lower positions (bottom right rail) and lower prices for premium advertisers. We devised a new model to match the real world data. This was used to produce the chart below. This new development certainly deserves the attention of most professional search marketers.
It took us several months to tease out the data and determine the factors behind this change. There were several keys to solving this puzzle.
Key Point #1: CPC and Quality Score Are Not As Closely Correlated As They Once Were
It's a well known fact in search marketing circles that higher quality scores mean lower CPCs and higher positions. That much is true.
However, there is another widely held belief that as you increase your CPC, your ads will appear in higher positions. This is where the traditional wisdom fails us because low quality scores (less than 7) quickly put the brakes on your ads. In 2009, you could try and muscle your way up the search results with high bids. In 2011, that strategy will probably only carry your ad to around position 3. Even then, your ad will appear less frequently and you'll miss out on greater numbers of impressions because you're competing with higher quality advertisers.
Higher positions are reserved for quality advertisers. It's never been more true than it is right now.
Key Point #2: Google Has An Inventory Problem
One of the challenges which lies at the heart of the auction-based advertising model is how to optimally extract the most value from advertisers. Historically, Google has excelled in this area. However, in 2007 they were facing a new problem - a supply shortage in commercially viable search terms.
While it may seem there are an infinite number of search queries available, the number of those which can drive significant revenue is relatively small (no more than a few million). This is because a search phrase needs two ingredients in order to generate search profits:
1. Consistent high-volume traffic
2. A large number of advertisers to drive CPC prices up (it is an auction after all)
AutoMatch was one method they used to address #1. AutoMatch works by triggering ads based on synonyms, misspellings, and so forth. Some love it, some hate it (I fall in the latter bucket) but there's no doubt it's an effective technology for increasing advertiser activity.
However, #2 was a problem. The number of active search advertisers has been topping off for some time. So much to the consternation of the smaller marketers, Google decided to reduce the number of ads shown per keyword by more than 50% and hopefully increase clickthroughs for the top advertisers.
It was an interesting experiment to be sure. I knew that there would be at least some success because the majority of Google's revenues were being generated from the top three positions in the search results. Showing fewer ads would mean that advertisers would need to increase their bids and/or quality scores or miss out on the shrinking bucket of first page ad impressions. Google was essentially creating an artificial shortage of inventory by limiting the the low-value ad impressions. The remaining ad impressions would then increase.
Eventually, they went too far. On July 17, 2008, Sergey Brin said:
"We've probably been too aggressive in reducing coverage in this past quarter. We now find ads are a significant addition to the page quality-wise."
Within a year, the lower page ads came back. However, Google was no closer to figuring out how to monetize them.
Key Point #3: The Top Ads Were A Sucker's Bet
It didn't help that many direct marketers had figured out the top spots were a losing proposition. Lower positions were often very affordable and generated a reasonable amount of traffic. If you wanted more however, you could increase your CPCs and/or quality scores (usually both).
The problem with this is that CPC prices increased in a more-than-linear fashion. At the same time, the higher positions also led to more traffic. The combination of higher traffic and higher CPCs lead to an unprofitable explosion in campaign costs. Smart marketers at the time figured out that they could maximize ROI by keeping their ads at the bottom of the page and leave the top positions for their less savvy competitors.
Rewarding smart advertisers for paying less isn't really a winning position for Google. What they've needed was an innovation and it appears that the breakthrough would come from devising a better way to price ads.
The Solution: Discounted Premium Ads
And herein lies the innovation. Rather than increase prices or reduce ads, Google decided to offer steep discounts for ads appearing in the top right rail or premium positions. The 49% premium for position #1 now appears to be gone. Advertisers can buy their way to #2 at most (and that can be difficult depending on the quality of one's competitors). To break through into the real traffic, you have to optimize your quality scores (7 out of 10 minimum).
The rewards are great for marketers who are willing to play by these new rules. Advertisers who earn their way into these positions can generate clickthrough rates of over 60% (that is not a typo). Furthermore, the average CPC drops significantly for these positions (about 50% in the chart shown above).
Don't think Google is giving money away. Far from it. They're making it back in spades due to the increased auction pressure in the lower positions. It's much harder to break through into the top positions these days and the increased number of advertisers down below raises CPC prices well above their equivalent 2009 levels.
This has a profound impact on search marketers: CPCs aren't as powerful as a lever as they used to be. Set your CPC to an acceptable price and leave it. Then focus on improving your quality scores.
The charts below shows our 2009 and 2011 simulations of a typical keyword for a direct sales website. In 2009, the premium positions were hardly worth it unless you were in a position where you needed to buy market share. Now in 2011, profits increase exponentially if you can consistently appear in the top premium position (more traffic for half the cost, get it?)
Changes to the PPC algorithms are always interesting but rarely pleasant. All things considered however, I welcome this change. It's a much simpler and more intuitive pricing model that rewards high quality advertisers and yet gives new advertisers (who typically haven't yet optimized) a fighting chance to get their campaigns off the ground. And those top spots - traditionally the playground of brand advertisers - are now fair game for the rest of us.
Did You Know?
Founded in 2004, AdGooroo is the original Search Engine Intelligence company. Our services help over 2,000 global advertisers excel in PPC, SEO, and Display Advertising.
Testimonials
“Based partially on your data, we have moved the site from a no-show, to 5th place, to #1 in a relatively crowded space in a couple of months.”
Scot Robnett
inSite Internet Solution
“Today, our use of AdGooroo tools sets us apart from most agencies”
Mike Lee
Director of Strategic Partnerships
DoubleClick Performics
Comments (17)
March 21, 2011 12:20 AM
Richard,
Man those days where you had to be on page one was a mind game. This is a great read. Thanks.
Posted by Jay Leishman | Reply to this comment
March 21, 2011 10:36 AM
Jay,
Well nothing has changed in that regard - if you aren't on the first page, you basically aren't participating in the AdWords auction (85% of all paid search clicks take place on the first page).
What has changed is that it the coupling between CPC, average position, and clickthrough rate has loosened up somewhat. This should make campaign optimization much more intuitive.
Thanks for your comment!
Rich
Posted by Richard Stokes in reply to Jay Leishman's comment | Reply to this comment
March 22, 2011 7:31 AM
Hi Richard
If I remember correctly, in your book you suggested that new advertisers bid low and aim for position 5 or below when starting out. You suggested they then improve their quality score before bidding higher to minimise the costs of moving upwards. Does this still apply or do you have different recommendations for new advertisers now?
Posted by Kerry | Reply to this comment
March 22, 2011 12:56 PM
Hi Kerry,
I would no longer recommend the strategy in the book. Instead, I would advise to calculate the desired CPC based on your conversion rate and set at that level. Once your ads have been appearing for a few days, take a look at your average position and work from there.
If your ad appears in the premium positions or the top two right-rail positions, then your quality score is likely high (you may be able to confirm this via the AdWords interface but this isn't always reliable). You should try lowering CPC to see how far down you can take it.
If on the other hand you are at position 3 or below, you'll need to optimize your quality score by A) optimizing ad copy, B) optimizing the landing page, or C) optimizing campaign/keyword structure.
Hope this helps and thank you for your question!
-Rich
Posted by Richard Stokes in reply to Kerry's comment | Reply to this comment
March 22, 2011 4:23 PM
Hi Richard
If I remember correctly, in your book you suggested that new advertisers bid low and aim for position 5 or below when starting out. You suggested they then improve their quality score before bidding higher to minimise the costs of moving upwards. Does this still apply or do you have different recommendations for new advertisers now?
Posted by Kerry | Reply to this comment
March 30, 2011 1:16 AM
Hi Richard,
Does this mean that even with a higher quality score we will have to pay higher CPC?
I have observed in one of my Campaign's that the average quality score of my adgroups are around 6 or 7 (one of my keywords has made it to a quality score of 10), yet I have to pay a higher CPC (between $2 and $3). And the average position of the campaign is 3.
I am not sure now how should I lower my CPC.
Posted by Z | Reply to this comment
March 30, 2011 11:28 AM
Thanks for sharing this research. If you are doing PPC it's important to keep up with the trends. Algorithms change fairly often enough and you don't want to get left behind.
Posted by Nick Stamoulis | Reply to this comment
March 30, 2011 12:38 PM
No - it means that below position 2, the relationship between CPC and quality score is somewhat decoupled. In this range, a lower quality score will translate to lower positions and less coverage (ie: your ads will appear less frequently).
At position 1 or higher, quality score makes a big difference. You need a 7 QS minimum but a 10 should get you a premium placements (there are essentially no 8's and 9's ... they are very rare). Increasing the quality score will definitely drop your CPC here.
Posted by Richard Stokes in reply to Z's comment | Reply to this comment
March 30, 2011 4:43 PM
I'm very careful about chasing Quality Score. I've worked on accounts both large ($3MM+/Month) and small ($5k/Month) and I've found that most of the time you're simply chasing a moving target. I recently inquired about shockingly low Quality Scores for keywords that have had steadily improved CTRs, dedicated landing pages (rich with textual content) and highly relevant (keyword-specific) ad copy. We made numerous adjustments over the course of a year and saw steady improvements to the aforementioned components of Quality Score, but we saw ZERO movement in Quality Score. We work with our Google reps daily and have an entire team dedicated to the account in question and they offered this as a response:
"While (Client Name)'s CTR of 2.93% for [Keyword X] may seem high, it falls short of Google's predicted CTR for that query, and therefore receives a lower QS. Other advertisers bidding on this term are also likely experiencing low QS as well."
Notice anything funny about that? Google has a "predicted CTR" that exists outside of ALL competitive CTR's. This means that Google can arbitrarily set a "predicted CTR" and inflate CPC's based on this undisclosed benchmark.
Bottom line, I agree with Richard, that the best thing to do is optimize creative according to business goals (Conversion Rate, CPA, etc) and aim to provide searchers with the most relevant creative/landing page for their particular query. Much like SEO, if the content is good, you can't go wrong.
Posted by Justin Terry | Reply to this comment
April 4, 2011 8:59 AM
Richard,
Can you elaborate more about the QS number shown in the AdWords interface as not being accurate? I know this is true - I have access to two accounts with two different advertisers bidding on the same keyword - both are shown as a 7/10 for that keyword but one is paying an avg. CPC of $.91 and the other is paying $1.58.
Posted by Aaron | Reply to this comment
April 4, 2011 10:00 AM
The example you cite is a good one but it's a bit fuzzy. I regularly see more obviously incorrect cases. Typically the quality score popup in AdWords will state something along the lines of "Your ads are not showing" and an explanation of some sort saying that the quality score is too low. However, it turns out that a different ad from a different ad group (almost always less relevant) is showing in its place.
What I believe is happening here is that the quality score and AutoMatch algorithms are jointly producing incorrect results. The QS algorithm has turned off the desired ad, while AutoMatch is substituting another ad in its place. Aside from the fact that this makes optimization much more difficult for the campaign manager, it also misallocates spend and conversions (outright reporting inaccuracies).
I wrote above "almost always" to soften the language here, but the truth is that I've never seen a case when AdWords has substituted a more relevant ad for a less relevant one. The quality score algorithm is simply broken. And if we're seeing it this often, it's likely that the problem is widespread.
Posted by Richard Stokes in reply to Aaron's comment | Reply to this comment
April 5, 2011 7:24 AM
Extremely interesting... Let's see if I got this right. So in effect you are saying that it's all about QS (CTR + "other factors") and huge ad spend now?
How many accounts were included in your analysis? Brand (own and competition) terms were probably included in the data?
I would argue Key Point #1, though. With a 7-10 QS you are in fact getting a huge CPC discount, so QS and CPC are still very closely correlating, with a bigger reward than ever before -> Key Point #1 "QS's impact on ad rank is greater than that of CPC bid"?
BTW, I find the ad position "0" in the charts very puzzling.
Thanks for a great post anyway. Keep them coming.
Posted by Toni Voutilainen | Reply to this comment
May 22, 2011 4:54 AM
Hi, Richard,
Thank you very much for this study. we semwatch.org translate your article in Chinese for our audiences working on SEM: http://semwatch.org/2011/05/google%E5%8F%98%E5%8A%A8%E4%BA%86cpc%E5%AE%9A%E4%BB%B7%E5%85%AC%E5%BC%8F%EF%BC%9F/
Some of our SEMers prove that your nowadays suggest is useful: to get right position first, then work on quality of SOME MOST IMPORTANT KEYWORDS, that is efficient way to work we think.
Thanks again.
SEMWATCH.ORG
Posted by semwatch | Reply to this comment
May 31, 2011 5:18 PM
Hi Richard:
>> a shift in the way CPC prices are being calculated as wel
Isn't that documented by Google? We have been doing similar research, and are using Googles public formula... and it makes sense.
-- your actual CPC = Ad Rank of advertiser below you / your QS
-- Ad Rank = Max bid x QS
-- http://www.googlehubris.com/article/googles-formula-for-cost-per-click/
Of course you can't calculate that on your own... you'd need data on your competitors Ad Rank that isn't available to you, but Google has all that data.
I think you could be right about "changes," but that could be done by Google messing w/ their QS standards (which is still a black box to us). If that is true, the "formula" stays the same, but one of the elements is forever "moveable" by Google... which would show varying results.
>> decided to offer steep discounts for ads appearing in the top right rail or premium positions
I'm not sure I buy that... not in terms of Google's "decision." However, if your CTR went WAY UP (because you were in 1st or the "above natural listings" position), it could have the same effect. Better position means higher CTR and QS, which means higher Ad Rank, which means better CPC by the same formula (whew!).
Our research show that *conv rate* is much more powerful in terms of ROI than QS improvements. This is true of CTR vs Conv Rate in ad optimization as well... a small change in conv rate is more powerful than a "medium" improvement in CTR / QS.
We're trying to push clients to completely ignore QS... it's Google's metric, not ours.
Good conversation.
Best,
Tom
Posted by Tom Doubt | Reply to this comment
July 27, 2011 5:31 PM
Richard,
I think that there are some slight flaws in this data. I look after an account which is well and truly a 'gladiator bidder' on all accounts i.e. short 2 month campaigns, very generic keyword in sometimes incredibly competitive industries.
For me there is no issue with being able to break into position 1 on about 85% of all keywords (and yes even the most competitive) just by bidding big against campaigns that may be CPA targeted. For me there is a lot of pressure to have strong positioning and this can be easily achieved through large bids.
The data is interesting though and I think it's great to look at the AdWords algorithm through the eyes of Google profit rather than 'relevancy' a term that they are constantly stressing.
Thanks,
Matt
Posted by Matt S | Reply to this comment
July 29, 2011 10:30 AM
Hi Matt,
What you're stating is consistent with what we're hearing from other advertisers. It is much easier to purchase premium placements on AdWords than it was say a year ago. This applies not only to CPA campaigns, but also to the more common CPC-based campaigns.
Gladiator bidding seems to have come back... and is there really anything wrong with that?
Posted by Anonymous in reply to Matt S's comment | Reply to this comment
September 11, 2011 2:55 PM
Excellent article, very useful to keep up to date with Googles algo changes. It would be interesting to know if the other top two search engines also conduct as many changes to help their advertisers - sorry I digress.
Once a premium ad position is reached the CPC is expected to be lowered, which does make sense as the CTR is much higher providing higher revenues for Google. However,
a) what place on the page is classed as a premium position?
b) how long does an ad have to appear in a premium position before a cut CPC is rewarded?
c) does the advertiser have to change the max bid CPC to benefit from the drop in CPC or is this automatically given by Google?
Thanks,
Mark
Posted by Mark | Reply to this comment