I recently grabbed a copy of the AOL search data that was released earlier this month. The dataset is rather large - 450MB compressed - over 1.4GB uncompressed in MySQL.
This data has been the subject of much controversy and has even resulted in at least 3 people getting fired. .
Anyhow, I've been parsing through the dataset and there's some interesting stuff in there.
I'll keep this thread updated with the information that I find...
- People use the search bar to type in website names a lot more than you realize. Out of the over 17 million queries, almost 3.5 million of them contain .com, .gov, .edu, or .org.
- People refine their searches as they go along
- dr shermam
- dr sherman longwood
- dr sherman vision therapy
- dr sherman vision therapy fl
- Babies search too! How do I know this? Witness searches for random characters or symbols. You can't tell me that people were actually expecting to FIND something like this:
- 6;p6p5p56ptpptptptppprprpprpprprprp...
G-Man
I think its important to remember that each of the instances of 'People' in the list items should be read as 'AOL users' I don't believe that neophyte data necessarily maps to the userbase of every site!
you're definately right..
It show's also in the top 10 searched women hehe.. Oprah was #1 and Whitney Houston #3 :P
https://www.seo-portal.com/aol-data-analysis-v...
6;p6p5p56ptpptptptppprprpprpprprprp... That was my cat on the keyboard.
I think if nothing else the data provides the first real chance we have had for a long time to confirm what we are told about the actions of typical se users.
The ideas on looking at the method of searching for a 2 kw phrase then refining strikes me as of marticular interest... if you look at phrases that are very competitive now who isnt gonna tag on a list of refining words to test?
Also the result of how the 1,2,3 etc word queries will be intersting - good work!!
I recently read that Google search does not use domain names unless you tell it to do so.
You stated "People use the search bar to type in website names a lot more than you realize."
Does that mean it is worth getting or buying longer domain names to match these searches?
no, what this means is that they are typing eg seomoz.org into the search bar. They know where they want to go, they just seem to be confused about which input field one should type a webaddress into.
the data is really interesting.. I posted several analyses already on the data here:
AOL Data Analyses
But Im still confused about the clicks for empty searches (see my last posting) anyone has an explanation for that one?
EGOL "I was very surprised an how many people do a search and don't click on anything!... or, maybe those folks clicked an ad that is not showing in this particular data".
That is my interpretation. Only 54% of AOL search queries are followed by a click on the organic listings. Most of the other 46% click on ads imo. AOL organic search results are often outnumbered by ads on the page and clickthroughs to these are not recorded in the data.
I have a tool based on the data which calculates traffic increase/decrease based on serps position which shows interesting results for certain changes in position. https://www.seo-blog.com/position-and-clickthr...
- Michael
Thanks Michael, I was really disappointed when I looked at these results and saw how few were clicking the organic SERPs.
BTW.... that is a really cool tool. Every person who does link building needs to play with that tool for a few mintutes before they sit down for a long link hunting session.
Here's a tool that I need.... I want to know if changing my title tag earns me higher CTR in the SERPs. I've messed around with adsense impressions vs organic clicks. Can you think of any other homebrew metrics?
I've seen the data and heard all the talk, but what really have we learned yet?
Well, I hope to post what I learn in this topic. Perhaps some others who have analyzed that data will share as well.
I haven't been focusing on general search habits as much as I've been focusing on the specific search habits of AOL users looking for the brands we work with... The information has been very valuable so far.
For instance, a client of ours has effectively built a solid brand name over time... Knowing that a popular brand tends to draw searches for the brand name, that's where I began to dig.
I was able to sort out any searches for 'brand name' which I could then tie to specific 'anonymous' users as well as the time and date of the search. From there, I looked into what ELSE each user was searching for in and around the time they were looking for 'brand name' (figuring most users did not limit their research to one search or brand).
This gave me TONS of great information. I was able to get an almost scary clear idea of what they wanted/needed this product for at the time of their search. Most users spelled out, through their search history, exactly why they were looking for 'X'.
I quickly saw people weren't just looking for a new 'X' simply for the hell of it... They were looking for a new 'X' for VERY specific reasons and/or to serve a very specific purpose.
Having concrete proof of these reasons and purpose is invaluable.
That is a terrific observation and worth following. I looked at that one time with regard to a visitor looking at my site and saw a very identifiable pattern that we've been familiar with for years. I'll relook at the data that way with regard to my site and those of competitors and see if the pattern holds up or if there are other identifiable patterns.
Thanks,
Dave
Ryan: I went through the exercise you suggested. It is incredibly revealing. As an old line operator of a brick and mortar local business we have learned this type of stuff after yrs of analyzing the sources of all our leads and sales and speaking with our customers...but this is better, more revealing...and yrs. quicker.
I'd suggest a lot of people do this. It provides killer insights that can immeasurably tighten or improve your websites.
Dave
Dave,
Sorry i didn't respond earlier (2-3 days late here)... i'm glad the tip helped you out. I was able to pinpoint trends i had suspected and trends i had not previously thought about. So as you say, it was incredibly revealing for me as well. It provides killer insights that can immeasurably tighten or improve your websites. Well said. Often times i find clients to be apprehensive about change. They see some success with their sites and tend to be hesitant towards site changes in general. As ridiculous as that sounds to me, i can understand their logic, while not being in agreement. That being said, i was able to take some of this data and show them exactly why i had suggested certain changes and what they have been missing out on. I instantly saw a lightbulb go on after doing so and now we're moving forward.
Ryan
ThomasB already has some of the breakdowns on the AOL search data you mentioned.
Thanks for the link sugarrae!
I was very surprised an how many people do a search and don't click on anything!... or, maybe those folks clicked an ad that is not showing in this particular data. What do you think G-Man? They have a great CTR from what I have seen in this data.
I haven't looked at the CTR rate on the keywords yet.
My limited experience on this is that Yahoo has a strong click through rate, based on running ads and not running ads when my site has a lot of relevant # 1's in Y. Do any of you have a feel or information on which engines see the highest ctr?
Dave
If I recall correctly on my sites it was MSN, Yahoo, Google in that order.
I've been roaming through the data. This isn't hard fact but there are tons of obscure search phrases that probably turn up crummy results, in part because the search phrases were too obscure or not clear enough. A lot of them refine their search terms and keep trying till they narrow the search. Certainly all the non-clicks do not turn into clicking on ads...a lot of them are repeated with refined searches...and then they click on a site, or an ad, or they do it again and again til the serps are better.
Just my humble opinion!
Good to see you blogging GMan. Hope you are getting strong!!!
I find the data fascinating. It is a teeny sample out of an engine w/ a small and diminishing size of the market...but a lot of data. I wonder how statisticians would treat it as an effective sample of search tendancies.
I suppose what you are doing, checking the one word searches and expanding them, especially if they involve many many searches give the best evidence of search tendancies since the samples are larger.
I've looked at some of the one word samples and run through them for many pages.
The one thing that strikes me is that it appears that so called popular one or two word searches don't dominate the volume of searches that include those phrases. It appears the long tail is far more powerful and varied than I imagined.
It makes me suspiceous of the data that overture or wordtracker prints out for the one or two word phrases. I wonder if they aggregate the long tail phrases to come up with the gross numbers for the one or 2 word data and then repeat the specific long tail numbers beneath their main phrase data.
Dave
I can't speak for how overture and wordtracker get their results.
The long tail is definitely long - I think that's why spammers have such success with splogs and/or scraper sites.
Increasingly people have resorted to longer searches than one word to find what they're looking for as single word searches are not targeted enough.