The other day I read a blog post that blew me away. Entitled "Google Bounce Factor Research Data Is In" and written by Visio, it was about the first real experiment that proves Google is using behavioral data in search engine rankings. Now, I've seen a few scattered posts where SEOs mentioned vague thoughts that Google might be tracking user behavior through either the Google Toolbar or Google analytics, but nothing concrete - nothing like this. I've even thought to myself on a few occasions, why would Google give away a tool as powerful as Google Analytics for free? And the answer screamed at me...For data-mining! But still, no proof or concrete evidence.
Let me give you a little background. A few days earlier, Visio had posted his reaction to something he read in Google's Official Blog proving that they use behavioral data in rankings:
"Similarly, with logs, we can improve our search results: if we know that people are clicking on the #1 result we’re doing something right, and if they’re hitting next page or reformulating their query, we’re doing something wrong. The ability of a search company to continue to improve its services is essential, and represents a normal and expected use of such data."
Ok, so they're starting to let the cat out of the bag. But to what extent is this in effect? Well, all (well maybe not all, but a lot) is revealed with Visio and his team's experiment. You will definitely want to read the entire post yourself, but let me summarize.
They did two experiments. The first, completely without the aid of Google Analytics, was to see if the number of clicks on SERP listings affected rankings. They had tested one site that was in a competitive arena with 100 participants and another that was in a non-competitive arena with 65 participants. Over a two week period they were able to alter the rankings of the non-competitive site, but not the competitive one. Their conclusion was that the number of clicks on a particular site from a Google SERP is part of the algorithm, but is only a small factor.
The second experiment was to determine the role of Google Analytics in the rankings. Over the course of several weeks, they had about 100 users click onto a site from the Google SERP and then spend from 30-60 minutes clicking around the site. They did no link building or anything else to it to alter its rankings. After a week or so, the ranking jumped 47 places from 80 to 33. In Google Analytics the bounce rate dropped significantly and the average amount of time spent at the site increased. After they ended the experiment, these numbers, including the rankings went back to their previously positions.
This experiment proved that Google was using the data from Google Analytics to improve the ranking algorithm. This experiment was on a small scale, but it creates a great jumping off point for further research. As far as I know, it's the first experiment of its kind. Hats of to Visio and his team for being so forward-minded to do such an experiment.
[I've given a very quick overview of the project and have not done it justice by any means, so please check it out for yourself to get all the details.]
This is really exciting information. Now we know for a fact that as soon as you plug that code into your web site, Google's computers are pulling in all kinds of valuable data on your website's visitors. At first it might seem sneaky, but honestly I'm glad that Google is using this data. This will actually make the results better for users.
Google just bought FeedBurner recently as well - imagine what kind of data they can capture from all the feeds of all FeedBurner's subscribers! Obviously a site with 10,000 readers is going to have more authority than one with 100 readers! I would say it's a safe bet that this new data will eventually find its way into the ranking algorithms.
Please feel free to share your thoughts, reactions, etc.
This experiment is really just the tip of the iceberg but I think these initial findings are incredibly interesting.
What I would be equally as interested to find-out is if the opposite would happen if you performed part one of the experiment in reverse. ie. take a reasonably well ranked site and have a team of people click in-and-out to increase the bounce-rate over a certain time-frame and see if this causes the site to drop in SERPs.
That is a good idea. I actually thought about it but never implemented it.
Your right its the tip of the iceberg but now that we do know the iceberg is there we can dig deeper and see where it leads.
"What I would be equally as interested to find-out is if the opposite would happen if you performed part one of the experiment in reverse. ie. take a reasonably well ranked site and have a team of people click in-and-out to increase the bounce-rate over a certain time-frame and see if this causes the site to drop in SERPs."
Right, wouldn't that just hand a significant weapon to everyone's competitors? Just get a distributed network of humans or bots whose job it is to click into and out of a specific site quickly for a given search.
If Matt's allowed to resort to his intuition, I will, too. [;-)] My intuition is that things that are out of a webmaster's control can not penalize a page. It's like the old truism that you needn't worry about being penalized by a spammy site linking to your page; just make that page doesn't link to a spammy site. In the same way, intuition suggests that anything a 3rd party could do to sabotage the rankings of a page should be devalued (if not outright ignored) as a ranking factor.
Exactly why Matts "intuition" went against it. It could have very negative effects if the idea got around that you could disable competition via this technique.
And BTW Matt doesn't have intuition. Anybody foolish enough to believe Matt is ignorant to this has to be pretty blind. Matt knows full well it exists. And that is why his comment, if it didn't exist he would have denied it.
Back to the second part. I have done a few very very small tests on the opposite theory that a high bounce rate can have very negative effects on your site and so far the results are all positive. The testing was way too small though to be totally positive yet, I am working on a larger scale testing.
There are a few aspects such as this and another technique that do not work so well in this user behavior world Google is entering. How Google will deal with these problems I am not sure but if it gets to the point were everyone is sabotaging each other Google will have to do something.
Also I have shut Analytics down on some of my big project sites, I can't have Google ruining my hard work because of Analytics data.
Hello winooski,
--- Quote from winooski ---
"What I would be equally as interested to find-out is if the opposite would happen if you performed part one of the experiment in reverse. ie. take a reasonably well ranked site and have a team of people click in-and-out to increase the bounce-rate over a certain time-frame and see if this causes the site to drop in SERPs."
--- Quote from winooski ---
Well, that's a good point. So, if I am offering my competitors the possibility to decrease my SERPs by unethical means, who tells me that they will not take the "opportunity"?
This said, they simply do a search on Google.com for a specific keyword or keyword phrase. They search for my website listing on my SERPs on Google.com. The "visitor" does not generate any clicks on my website and the "visitor" leaves my website after a couple of seconds.
You repeat the same procedure for x-times by using a different IP address.
Due to the collected data using Google Analytics, Google.com might think that my website is not "relevant" or "interesting" enough to be highly ranked for a specific keyword or keyword phrase.
Thus, I get "punished" and my SERPs on Google.com will drop. On my opinion, this is much too risky and dangerous. My 2c.
graphically & sincerely,
Marc Klein
mediaVinci - The art of invention
- Casey Removed Link
... seems like this is a day where nothing is really working. I can no longer send emails, I cannot access my root via ftp/sftp, I cannot access the administration control panel of my server ... puuuh ... and clients are waiting for some files ... what a hard day ...
Regarding feedburner:
>>I would say it's a safe bet that this new data will eventually find its way into the ranking algorithms
I'd agree. It will be help identify splogs... low readership, heavy on the ads, short domain history... might smell like spam. Feedburner + Google Reader + Google Analytics has got to give google a decent picture on the readership question.
There's something beyond this most people seem to be missing...
If Google are ranking sites higher, based on current authority, it'll kill new sites.
We'll see wikipedia effect everywhere. The top sites will all be heavyweight ones, with no smaller stuff getting through, even if it deserves to.
A slippery slope...
Exactly. By the time a genuine Authority site (that isn’t concerned with SEO) accumulates enough status with Google to become discoverable by users (if it ever does) it may be obsolete. Already Google is probably not the best place to search for this kind of fresh authoritative information. I’ve noticed that searchers are often directed to an index page, which has higher page rank instead of to another page, which is clearly more relevant to their query. I wonder how many people on here are often using an alternative search engine because of this. This may actually be good news for SEOs but in the long run it will be bad for Google.
May want to look out for something I'm writing atm. I'll message you when it's out.
Hello Dr Dave,
--- Quote from Dave ---
I’ve noticed that searchers are often directed to an index page, which has higher page rank instead of to another page ...
--- Quote from Dave ---
This is a phenomenon I have noticed as well for many times. As far as I know "Premium Google AdSense" Advertisers can adapt and customize the look & feel of Google AdSense advertisements in such a way so that an "outside viewer" will no longer notice the difference between a simple textlink and a paid CPC advertising link.
My 2c,
" I create, therefore I am "
Marc Klein
mediaVinci - The art of invention
- Casey Removed Link
You lost me...
Just make sure you sign out of Google account or turn off personalized search (or the &PWS=0 trick) before you check rankings after doing a click experiment like that on your own site ;-)
Oh I can just see all the new "services" like traffic exchange breaking into the new niche, search and explore services - people paid to do Google searches and click on specific results. *Sigh.*
I experiment under one IP and check results on another. As far as Google is considered they are totally different people.
As for webmasters clicking for rankings I believe that is highly unlikely to go far. I believe that clicks can effect to a small degree but only in long-tail type searches, it would take thousands, tens of thousands of clicks I believe to move a ranking up a few spots for a competitive keyword using clicks.
Google analytics tracking has a much higher value.
I mentioned this in another post but Google has the ability to collect massive amounts of data. It's more than just Google Analytics. With the Google Toolbar, Adsense, Blogger, YouTube, and other major web properties including Google itself I think Google knows more about what we click on than we give them credit for. I'm sure they are using this data to improve rankings.
I'm headed to read the full version, but thanks for the work Visio.
Why wouldn't they be using what they have in their stable to help measure? The only thing better would be do have every user complete a follow up survey after their search rating their results, satisfaction and suggestions/comments. Since we all know that won't happen ... Google will pay attention to every piece of data they have, that's just common sense to me. The draw back is you might have to have Google Analytics in some cases to help your SEO cause ... I think that's no good.
Very interesting expirement. Thanks for bringing into attention.
This isn't too surprising... I mean, if you have this data, and you can find a way to make it useful to better your service, wouldn't you use it?
Obviously Google is cautious and probably working to best determine how to not only use this information better, but also to minimize the ability to be manipulated by it. And of course, I would have to imagine that it would be heavily down-played publicly for these same reasons.
I'm sure as much energy is being put into protecting against manipulation as in how to use it to better results, which I would imagine many of the other metrics used for ranking can be called on for this.
I would imagine that the best way to protect against manipulation is to not allow any one metric to be too powerful, so that it isn't just "the highest score" which wins, but also takes into account the relative scoring across the metrics so that a high score in one area but low scores everywhere else might be suspect, whether it is click-thru and bounce data or anything else.
Wow...
I'm with roadies...should we keep analytics on our sites?
I guess conversions from our landing pages are even more important now, if it effects our organic ranking also.
Thanks for this insight WebGeek! It has always been a "I wonder if..." type of scenerio that I (like a lot of SEOs) assumed was being used, but no hard evidence. Even on a small scale, I'd agree there is evidence here that the data being collected is used in their index.
Now the question is: do I keep analytics on my site, or take it off? Hmm.... that's a toughie.
You're welcome roadies! This was exciting info - too good to not share. :) I know a lot more research has to be done, but Visio and crew have some other great experiments in the works.
Visio would probably be better to answer this than me, but it sounds like (at least in the short term) if you have a high bounce rate, you might be better to take it off, but if you have a low bounce rate and people spend some good time at your site, it will help your rankings.
Personally I'm of the opinion that even if it temporarily hurt rankings, it would be worth it to keep Analytics installed, because it will be a good indicator of areas for improvement.
I do not not think the this experiment proves that Google uses Analytics data without a doubt but I do think that it is good evidence that Google is using that data to change their SERPS. At this point I think that it is only on terms that are not searched that much. I have also seen evidence of this. I have only one site using Google Analytics (one of my lowest traffic sites).I have been looking at each keyword (all are longtail - none are competitive) that people have used to find the site and without fail all of the terms that have a high bounce rate have been repositioned to much higher rankings (most were between 60 and 500). Those that have a very low bouncerate are mostly on the first two pages.
BTW, I have removed analytics from the site and am testing to see what happens now.
Interesting thought about the FeedBurner purchase and using those stats for Google rankings. Thanks for the post.
Oh, hey wait a second.
Matt Cutts yesterday rasied an issue about HitWise:"Hitwise captured 4,201 unique terms sending visits to the website.”
From SEOMoz today:"Google is Using Behavioral Data in Rankings" And "Similarly, with logs, we can improve our search results"
Aren't these essentially the same thing... 2 companies using traffic data to improve a product?
ANNOUNCEMENT!!!
I am working on another test experiment which is related to this. So far very interesting stuff happening but not enough conclusive evidence yet. I will let you guys know when/and if I do find evidence to support this arguement. Sorry I can't provide any more details.
That's great Visio. Good luck with the testing. I hope it scales for you too.
I've wondered how much benefit Google has gained from Analytics implementations. It seems like they would have been able to do much of the same sort of thing pre-Analytics using data collected on sites serving AdSense, which already represented a large cross-section of the web.
Interesting... My colleague and I always though this might be the case!
Well this way I guess google want to control the Web, which maybe not ethical in sense that peple would not like to track their behavior on internet
-Stella Roy
That's What i write but nobody seem to bother that :(
Read it & plz let me know what you think.
https://www.squidoo.com/predictions
It sure is an interesting experiment and I don't blame Google, if true, that they use all data they have to improve the users search experience.
Though, I'm not sure if I read the second experiment right. The users click in from the SERP, same as in the first experiment, and then browse around. Than you have to possible factors that could improve the rankings. I would think that the second experiment would be more successful if the users came direct to the test site so that Analytics would be the only Google tool to possibly affect the ranking. Saying that the movement in rankings was higher doesn’t bite, it’s easier to move up 40 positions way down in the SERP than it is to move a couple of positions on the first page.
Matt: I don't see you denying the fact that this could be true rather that the experiment isn't perfect. I remember at SES NY when a colleague of yours spoke about how you use Google Toolbar to determine Adwords relevance.. I don't see how this would be any different.
Okay good question and point.
See I believe a sites overall bounce rate has a factor however this I have not been able to totally prove.
However if a user finds your site by doing a search, and the overall users who do that search and end in your site seemed pleased(do not bounce) and spend a good deal of time at the site it tells Google the site might deserve a higher ranking. However if the users just go direct to the site Google doesn't have much to attache to ranking.
I hope that made some sense :P
Matt can't quite deny it because there is too much data supporting it AND if I were you i'd read the patent on this and the privacy policy.
Google probably knows about this... how do we know that some weasels have not been clicking the wrong links?
Sorry I didn't get that. ...yeah I need sleep :P
EGOL, I have a feeling you know more about this topic of Google and user data than you let on. :) I remember reading your comments on the Search Engine Ranking Factors a while back. Any thoughts?
Posted here two years ago - in August of 2005
https://www.seomoz.org/blog/throwing-out-the-book-on-seo
Wow, I'd say you called it a long time ago. That post is hidden gem.
[ Live link: Throwing Out the Book on SEO! ]
You were the one who got the wheels rolling for me, definitely has lead me into some interesting stuff. So thanks Egol! You probably should have been the one doing the article ;-)
Hello,
I was wondering, has anyone tested if the Google Analytics script has any positive/negative incluence on the SERPs on Yahoo.com, live.com or msn.com? This would be quite interesting to know.
Well, Google and Yahoo, MSN, Live are not "best friends". They are severe competitors in the search engine market.
I have also noticed some negative speed effects on the page rendering of my site for a couple of times using Google Analytics.
Any thoughts and comments on this topic would be welcome.
graphically & sincerely,
Marc Klein
mediaVinci - The art of invention
- Casey Removed Link
There is no way for Yahoo or MSN to get the data Google can see in the Google Analytics accounts... as for them being able to track it themselves Y! and MSN currently do not have that kind of technology... Google is still tweaking theirs so it will be a few years before yahoo and msn even figure out what is happening :P
Hello Visio,
thanks for the clarification. Indeed, this would have been my second question, if Yahoo or MSN are in possession of a similar technology.
I personally do believe that Yahoo and MSN is totally aware of what is happening at the moment. And the more people are joining Google Analytics, the greater the fear for Yahoo and MSN to loose a competitive edge in the global search engine market.
Thus, I would be a real fool and naive to believe that the inclusion of the Google Analytics script has no influence at all on my SERPs on other search engines.
Furthermore, I get more and more the impression that I am taking part in a "huge experiment" by having joined Google Analytics. Even Matt Cutts and the entire Google Quality team is not aware of into which direction this "trip" is going to. And I do no longer see any reason, why I should "play around" with the valauble data and the privacy of my website visitors and customers. Thus I have removed all Google Analytics Scripts from my website "PIXEL TEMPLATES".
As said, for me personally, it is way too risky.
Google's invention of "Link popularity" has already evoked a real mass hysteria these days. I get more spam mails to join and buy links on directories than spam mails for buying replica watches. Every "wanna-be" internet marketer is telling you to skyrock your PR by ordering links on other websites. The more the better. On my opinion the entire "Link popularity concept" is already a big failure.
And, most of all, search engine providers have to realize that the entire CPC advertising model has become a dead end road. One would have to "invent" too many tricks and techniques to prevent click fraud. And all these measurements will more and more evoke a direct conflict with privacy related concerns. I mean, a simple IP address tells you a lot these days. Thus, search engine providers have to face the simple truth. It is impossible to fight click fraud without infringing privacy related conerns.
My 2c,
" I create, therefore I am "
graphically & sincerely,
Marc Klein
mediaVinci - The art of invention
- Casey Removed Link
Do you think that similar data can be obtained through an adsense script?
Hello EGOL,
that's a very good question.
You need some hardcore techies and real geeks to find this out. And you need some "knowledge" and information on the structure of the backend system.
graphically & sincerely,
Marc Klein
mediaVinci - The art of invention
- Casey Removed Link
Color me disbelieving. Any time I read something like "This experiment proved that Google was using the data from Google Analytics to improve the ranking algorithm" I get really skeptical. Not just because of words like "this experiment proved," but because it doesn't fit with my experience or intuition.
but our SEO-spidey-sense was tingling...
it was...
honest!
I'd expect Matt to say that. Essentially what ever you discover, or think you discover about how Google ranks sites. Google will never ever give you a definative answer.
Personally I sit on the fence on the issue: If no one knows the Algo how is that different to everyone knowing it? The ranking will still be the same.
No one knows the algo fully, we know parts of it, putting parts together you can get a pretty good picture of what is going on. Being skeptical is natural, but please test it yourself. If you use Google analytics take a look at your bounce rates for various phrases, I know you will notice some very interesting stuff.
Alot of data I have gleaned from various analytics accounts was not mentioned due to the fact that in those cases the sites were active and there were many factors that could have effected the rankings even though it was unlikely that was the cause for the great changes.
Matt,
I can see where it would be less than ideal for Google to use Analytics data to reward/punish the subset of sites, which use it. But, is it not true that bounce rates, and time spent on a site could be seen (by Google) as factors which indicate the quality of the search results.
We're feeling around it the dark here; sometimes apparent cause and effect are going to be misperceived.
You almost sound unbiased Matt :P
I am sorry but you are biased. You stating your "skeptical" instead of actually saying I am wrong almost in and of itself proves that I am right. You have the data, you know what is going on. Why don'y you just plain out say I am wrong?
I have anecdotal evidence that Google uses their toolbar to influence rankings.
I had 2 personal websites with brand new domain names indexed in less than 2 weeks by doing nothing. These were new registrations, with zero links and Google still found them.
The only way these could be discovered is by google scanning all the ports on my server or through the toolbar.
I have no doubt that Google is using the Toolbar to gather data and influence rankings. There's another possibility that could explain its quick indexing as well. When you register a new domain, often AboutUs.org creates an automatic entry for that site, and a link to it. Google may have found it through that site. Take a look at "https://www.aboutus.org/YourDomain.com" (switch out the "YourDomain.com" for yours) and see if there's an entry already, or if it says something to the effect that it's creating a new entry.