There’s a dirty little secret among Bing (Microsoft adCenter) PPC managers. Back in the days of low traffic and dirt-cheap clicks, many of us let our broad-match keywords run wild. Performance was good, and we wanted all the cheap traffic we could get, so what was the harm?
Then Came Quality Scores...
Over the last couple of months, as Bing search volume steadily increased after the Bing/Yahoo integration, Microsoft announced its own version of quality scores (which they somewhat mercifully just called “quality scores”), and those scores started to appear in adCenter. The PPC community initially responded with an enthusiastic yawn – we wanted to care, but just couldn’t quite pull it off.
Measuring a thing has a way of changing it, though, and so I thought it might be time to collect some data. I started to do something that honestly scared me to death – pull back my big broad-match ad groups in Bing to tighter phrase-match groups.
Initial data was promising, but today I came as close to waving a magic wand as I’ve ever seen in paid search. These are the numbers for a small, tightly-focused Bing ad group:
Although all of the keywords were set to broad match, each 3-word phrase only differs by the last keyword (I’ve removed the actual keywords for client anonymity). Keyword-relevance was high – search query reports showed mostly long-tail terms based on these phrases – and CTRs were strong. Landing page relevance showed no problems, but quality scores were stuck at 5s.
So, I switched all 3 keywords to phrase-match. As soon as I hit [Save], with no new data or time passing, I saw this:
Same keywords, same timeframe, and quality scores instantly jumped from 5s to two 9s and a 10. Astute observers may notice that my historical CTRs for the period went up. I have no explanation for this – I double-checked in disbelief and was able to replicate the shift.
But Does It Matter?
Ok, so quality scores went up, but Bing currently claims that this is just informational – unlike Google AdWords, quality scores don’t impact position or pricing. Still, it begs the broader question: will the tighter match types hurt or help performance?
While the data above is from today, I’ve been making similar changes for a larger client (removing low-QS keywords and narrowing broad-match keywords to phrase-match) over the last couple of months. In early June, I overhauled one ad group – measuring 1 month before and 1 after, I got the following results:
CTR jumped dramatically, average position improved, CPC improved, and CPA dropped like a rock (except that when the rock landed on the client’s head, it turned out to be a wad of cash).
Honestly, I didn’t believe it. In the interest of total transparency, the client had undergone a major offline advertising push, and I figured this was simply lucky timing. So, I tried it again.
Here We Go Again...
A couple of weeks later, I rolled out similar changes on another large ad group. At this point, the offline ad-spend changes had settled in a bit. Again, comparing the month before and after:
Although some of the results weren’t quite as dramatic, the overall impact was still extremely positive. This is, of course, anecdotal, and conditions may have varied across the time periods in both cases, but I’m rolling out broader testing, because I think the data is compelling.
What Should You Do?
I’m not going to suggest that you instantly change all of your broad-match keywords in Bing to phrase- or exact-match, but I do think that you should start taking match-types and quality scores in Bing as seriously as you do in Google. Microsoft is still trying to figure out where to take quality scores, so there’s no time like the present to experiment. Pick an ad group and:
- Remove any low-QS keywords with very low volume
- Review your Search Query Report for keywords driving clicks
- Switch all broad-match keywords to phrase- or exact-match
- Add in any additional keywords turned up by the query report
- Turn it loose and start collecting data
Obviously, if you see anything close to the results that I’m seeing, start rolling changes out to other ad groups.
For the moment, Bing quality scores are a bit crude (and probably too easily manipulated), but I suspect they’ll advance quickly and we may be taking them a lot more seriously in the next few months. I also think that we Bing PPC managers just got a little lazy. Broad-match worked, so why fix it? If nothing else, it’s time to take a hard second look at our tactics.
Could the change in historical CTR be because when you had broad matches set, Bing was looking at the CTR for all search terms that caused ads to appear and when you changed to phrase match they started showing CTR only for phrase matched terms?
It's possible, although it worries me a bit that historical data can change. This should be actual data, not a keyword-tool style estimate. The only other thing I can think is that this particular data included the current date as the last date - maybe the real-time data calculation was impacting the overall numbers?
I didn't want to keep switching back and forth from broad-match to phrase-match and vise-versa on a bunch of ad groups to see what happened. I did it a few times and then got paranoid :)
I think Ben is exactly right. We recently rolled out this entire, exact process for an Adwords campaign, and got virtually the same results as you Dr. Pete!
Yeah, it's definitely worrying if my speculation is correct - you want to see the actual data collected, not what Microsoft thinks is the best data to see. It's a good way to encourage bad practices, with people just switching math types rather than creating a new keyword with a different match type and testing the results.
If your keywords had been running on broad match for more than a few days, I can;t see real time updates having that much of an effect I'm afraid - especially on quality score. Sorry to increase your paranoia ;-)
Awesome Post Dr. Pete. I tried you tactic and it worked. Quality score of some of my keywords is 10 now. Thanks a lot. Hope to see more from you on PPC.
Excellent - thanks for the additional data point. It'll be interesting to see how quality scores impact performance going forward, but it definitely seems it's very sensitive to match-type at the moment.
Hi Dr. Pete,
What about the impressions and actual number of conversions? Were the changes in these areas negligible? Some clients are willing to pay higher CPA for higher volume(conversion). 38% drop in CPA sounds very good if your conversions did not drop by 38% as well.
I think no client will be willing to pay more than the Max. CPA as long as he knows what does that means. When you pay more than the Max. CPA you no longer make profit on sales. Part of PPC optimisation is to make sure that your ads and keywords meet your CPA goals.
While I personally agree with your philosophy, I've had clients where "Max CPA" varied pretty wildly, depending on how many leads they wanted that month. Some clients want to run campaigns very aggressively and focus primarily on total leads.
Good question - I was originally toying with reporting some raw numbers, so I didn't want to reveal volumes. Once I switched to %-change, it didn't dawn on me to add it back in. In the first scenario, clicks increased +134% and conversions increased +640%. The dramatic change (even unbelievable) in these numbers was the reason I suspected the offline advertising impact and re-ran the test.
The results in the second test were less impressive but still solid - +18.1% clicks and +27.9% conversions. Of course, these were running over 2 time periods (no good way to A/B test it), so there may be confounding factors. I think the overall picture is compelling, though.
Great too see some PPC content on here, I only dabble in a small amount of PPC these days for SEO testing or brand reputation management.
Very interesting analysis on bing quality scores I did not know bing was lifing their game in the Quality score areas.
Too bad in the australian market bings market share is still not that volume heavy but =( in the US it seems to be a very different story.
The usefulness of Bing in the USA may be nothing to do with market share - the Search Allinace actually seexs to have about the same share of the market in 2010 in Australia than it did in the US, 30% vs 29% - and more to do with the population of the USA being 14.4 times that of OZ according to Wolfram Alpha. This means that this share represents 14.4 times as many searches in the US than in Australia - quite a difference when it comes to potential visitor numbers.
Cool tool the Wolfram Alpha...
Yes that is very true the US is 14 times larger yet another problem is that Google is a far stronger player in the AU market...
Wolfram Alpha is a very cool tool - or "knowledge engine" as they describe themselves. It's built on the back of Wolfram's Mathematica product, which is a sophisitcated mathematical toolkit and programme that I used back in my uni days for solving all sorts of things. They've taken the power of that, added data sets from all over the world on all sorts of things, from populations to food nutient numbers and come up with something very interesting. The difference between Alpha and, say, Bing is that when you ask Alpha a question it comes back with a direct answer - one reason Bing have incorporated Wolfram Alpha into their universal search results.
I am a bit worried about the information from that second screenshot because it doesnt match with what the article says. Eg. when it says :So, I switched all 3 keywords to phrase match. As soon as I hit [Save], with no new data or time passing, I saw this...- [image ] if the keywords are phrase match, why does it says Broad in the match type coloumn?
Eek - thank you. To fit space requirements and protect the client data, I had to edit the images, and apparently I over-edited that one. I've fixed it now.
This is just another reinforcer that we should always be testing ideas. -1/3 CPC is amazing for one little tweak!
It's good to see some substantial evidence behind these factors as I have been testing changes between broad and phrase myself. Hopefully AdCenter will make the big changes in their impacts more sound in the near future, and not as "speculative".
Very interesting study. I need to start testing out making the changes in Bing (Broad to Phrase) to see if I can get similar positive results. How many tests do you think is necessary to tell if you should do roll outs to other ad groups?
It really depends on your comfort zone. I started with one client's largest ad group, so now that I've done 2 or 3, I'll probably just roll out the rest all at once. The nice thing with PPC is that, as long as you keep good records, you can always undo short-term mistakes. Just log what you did and keep an eye on the data. It may take a month to get solid results, but you'll know within a week or less if something's gone wrong.
Hey, just had a question about this comment "Bing currently claims that this is just informational" do you have a source for this? I'm interested to learn more about their official stance on this.
Sure, here's a resource from Microsoft. Actually, a very low quality score could mean your ads aren't showing, but QS doesn't have the same impact on price/position that it does in AdWords:
https://advertising.microsoft.com/small-business/product-help/adcenter/topic?query=moonshot_conc_aboutqualityscore.htm
This is compelling enought that I think I'll bring a few campaigns into Bing to test again... Does anybody know of a quick+free way to import adwords campaigns into Bing (for the lazy among us?)
If you're on a PC, you can export/import from the desktop apps for AdWords and adCenter. They're free and work pretty well. I'm not sure if there's an adCenter desktop for Macs.
Did you see an increase in total conversions? What about the bottom line? Was there a drop in traffic and if so...was it detrimental to the bottom line?
Great post, love the PPC content as I do both :)
See my reply to @seowebjunction above - the short answer is "Yes" - clicks and conversions were up in both scenarios. In the first test they were up a lot, but I suspect some outside factors were involved. In the second test, total conversions increased about 28% (the number of leads, that is, not conversion rate). The performance improvements did not cost me traffic, which was my initial fear in narrowing my Bing keywords.
Great post, thanks. Do you know if Bing normalizes the CTR based on positioning like Google does? Google essentially takes account for what position your ad served and how that would affect CTR when it reports CTR...so if you have a poor CTR, it's not because of position. Does Bing do the same?
I honestly haven't seen great documentation on how Bing determines ad position, and they're not being clear right now on whether or not their quality scores factors in (or will, down the road). This is one area where I've actually found Google more transparent, but I admit that I also no the AdWords program better, when it comes to the details.
https://itechbee.blogspot.com/2011/08/square-now-processing-4-million-in.html
Whoa! 33% decrease is cost! Great tweak!
Careful: as mine and Pete's discussion above hints at, when numbers look too good to be true they usually are
Is doing advertising in bing a good idea? will it have a better impact compared to Google Adwords? I mean , so far i have been using Google Adwords and i am seeing a very significant impact from it... Can anyone with experience with both of them give me some clue about which one is better, and what is the pros / cons of using one againts another... (or if you would be so kind to point me to the right direction by providing some useful link backing up your point) :D
I have found Return on Advertising Spend in case of Bing very high (up to 3000%) even when we are not providing any special offer or discount. You spend £100 and can earn £2500+ . Not bad. The downside of bing PPC is that you don't get as many clicks as you can get through Google esp. here in UK where Google search market share is close to 90% .But still i always make sure that i allocate some budget for Bing PPC cos of it excellent ROAS.
Hmmm even with only 10% search market share, bing could perform this good... ?? that is very interesting O.o;... thanks for sharing...
I get a better ROI from bing than with google. the traffic is generally excellent quality in my experience, better than google. i think a lot of people use google just to browse competitor sites and cost them some money on PPC ads. well, thats my theory as to why bing does better for me. of course, the search volume is much less, but enough to be worth using it.
I agree, I do get a better ROI from bing, and the traffic is better quality, but the volume is no where near as high as Googles, so less sales.
I have recently tried moving alot of keywords over to phrase match, on adwords with similar results, although not as big a jump in quality score as bing, but its definitly having a positive impact.
@Lethalor and @Rick, thanks a lot for sharing... this is very interesting , despite having much less search market share, that many would find bing comparable or even better then google ads .... (again , thank you SEOMOZ for putting a lot of great internet marketter in one place sharing their thought) xD
Yes, I have noticed that Bing converts better. I think it is because they are actually getting better quality traffic then Google these days.
I'll just add to what @seo-himanshu and @lethal0r said - Bing ROI is often better than Google. It used to be that volume was very low, so you just looked for a sweet spot and spent what you could. After the Bing/Yahoo integration, volume has increased significantly. I think Bing is well worth a look.
The deciding factor for me is usually budget. You have to have enough to spend that:
(1) The time spent managing 2 campaigns is worth it
(2) You can collect sufficient data in Bing
I wouldn't spend $1,000/month on Google and $250/month on Bing, personally. Better to consolidate your efforts and data.
thanks a lot everyone.. hmmm very interesting...
Bing / Yahoo integration?? i just know this O.o; .. you are saying that if i put ads on Microsoft Ads, it will also show on Yahoo?
This is extremely interesting, i will definitely try it tomorrow (hopefuly i could find the same performance in my country, since in Indonesia no one really use bing)
Yes, adCenter (Bing PPC) now powers Yahoo search ads (as Bing's engine powers Yahoo organic search). So, the actual market share is closer to 25-30% here in the US, I believe. I'm not sure on the exact stats/timing for Indonesia, unfortunately.
It's nice to see PPC posts especially AdCenter related, as for the test you just did i agree Phrase Match will give you better CPC and probably better CTR.
I have tested this with 2 separet ad groups for the same keyword on ad group with broad match, and one ad group for phrase match. Phrase match is performing better without a doubt with significantly higher CTR but with about the same CPC (phrase match a bit lower). I must say i have the same Quality score 9/10 for both phrase and broad match in separate ad groups.
In conclusion Phrase match is performing better than Broad match but if you need to get more impressions and be visible for more keywords you need to use broad as well.
Interesting post and great to see the CPA decreases. Just wandering what the overall impact was on total sales volume?
See @seowebjunction's comment above. Clicks and conversions increased in both cases, although less so in the second case. I suspect there was a confounding influence of offline advertising in the first test.
WOOT! Getting more clicks for less $$$!
Tahnks for the info! I have been wanting to fireup the Bing Adcenter lately. I will keep an eye out for this.
I've got a few campaigns I'd be interested in trying this on. I'll try to remember to let you know about any positive changes.