Yahoo!, through both Site Explorer and their advanced link search operators, provides the most robust link data currently available on the web. Despite this offering, and in spite of some excellent blog posts and resources on this subject (1, 2, 3), it seems that not everyone doing SEO professionally is familiar with how to use Yahoo!'s data to the fullest extent. Even worse, we might not all be aware of how critical it is to measure, quantify and review links. This post is going to take a stab at solving some of these issues and determining if there's more that needs to be done. Let's start with a quick poll:
Now that you've voted, let's explore what Yahoo! offers in terms of link data. To start with, they've got two ways of accessing links - via Site Explorer, which provides a unique interface for browsing link data, and the normal web search engine, which lets you apply many more advanced parameters and modifiers to your link-based queries.
Yahoo! Site Explorer:
There's only a few basic features here, which includes the ability to see:
- A list of links that point to a given URL
- A list of links that point to a given fully-qualified domain (e.g., southernfood.about.com, www.seomoz.org, or reddit.com)
- A list of links that point to a given pay-level domain (e.g., *.about.com, *.seomoz.org, or *.reddit.com - including all their respective subdomains)
- A modifier that removes links coming from internal pages, either on the fully-qualified or pay-level domain
Using Site Explorer is simple - just enter a given URL and go (or type a standard link or linkdomain search query into Yahoo! Search and you'll be re-directed). You'll be given options to modify the domain parameters and exclusions in the results.
Yahoo! Site Explorer Link Search for NBCOlympics.com
The biggest weakness is the tragic lack of crucial data pieces, including:
- Ordering - the links are given in "no particular order" according to Yahoo!'s public representatives, though SEOs generally believe that they tend to show more important links before less important links (but certainly not in order from most to least valuable/popular/important/etc)
- Nofollow Included - in a tragic move, nofollow'd links are included in the list with follow'd links, and no differentiation exists between the two, forcing SEOs to do their own research page by page to find which links the engines might actually be counting.
- Target URL - unless you choose link to only a given URL, you don't get to see which page on a domain/subdomain a particular link points to
- Anchor text - no anchor text is provided to show the linking term/phrase/alt tag that pointed to the page
- Importance metrics - no indication of how valuable/important a particular link or domain might be is provided. Obviously, Google's PageRank would be a strange one to show here, but Yahoo! used to have their own link graph based value - webrank - which was quickly discontinued (way back in 2004, I believe)
Despite these weaknesses, it's at least valuable for browsing through a site's links and attempting to get rough information on the types of sites and pages pointing to the URL/domain. If you want more, you can always click through to each individual link for some of this additional data (though this is a time-consuming process).
Yahoo! Search:
Despite the fact that Site Explorer is meant to be the flagship link searching product, Yahoo!'s normal query system actually provides far greater functionality. This includes the ability to get:
- A list of links to individual pages or entire domains
(link:https://www.yourdomain.tld) or (linkdomain:yourdomain.tld) - Lists that exclude pages from certain domains
(-site:domainx.com) or (-site:.co.uk) - Lists that exclude pages with certain keywords
(-keyword) or (-intitle:keyword) or (-intext:keyword) - Lists that exclude pages/domains with certain attributes
(-inurl:keyword) - Lists refined to certain domain extensions
(site:com.tr) - Lists refined by geographic region
(region:europe) - Lists refined by keyword inclusion
(+keyword) or (intext:keyword) or (intitle:keyword) - Lists of pages that contain links to other pages or domains
(linkdomain:domainx.com, linkdomain:domainy.com)
For a full list of these modifiers and how to apply, I recommend Ann Smarty's brilliant post at SEJournal - Advanced Link Operators to Explore Your Competitor's Backlinks.
Using Yahoo! Search to explore links requires modifying the standard link queries. If you simply use a query structure like "linkdomain:nbcolympics.com" (to see links to a domain) or "link:https://www.nbcolympics.com" (to see links to an individual page), you'll be redirected to Site Explorer. You need to combine these queries with additional parameters in order to get data from Yahoo! Search.
Yahoo! Search query for links pointing to NBCOlympics.com that contain "Beijing" in the URL
The weaknesses here are the same as those for Site Explorer - no link targets, no anchor text, no importance metrics. I won't completely re-hash, but the only real advantage over Site Explorer is the ability to refine the query, and the only weakness (from Y!SE) is the lack of a TSV export function.
How I Use Yahoo!'s Link Data:
To me, links are a fundamental part of SEO, and inseparable from the optimization and analysis process. Since Yahoo!'s data is the most consistent and valuable currently available (as Google made their link query useless in 2003 and MSN disabled theirs earlier this year), I'm a very heavy user for tasks like:
- Determining the approximate number and strength of a site's links (to help uncover what role link strength/weakness is playing in rankings & search traffic)
- Tracking relative link growth/shrinkage over time (this is why our SEO Analytics tracks both Yahoo! & Yahoo! Site Explorer numbers)
- Uncovering links that could be causing potential problems (paid links, spammed links, low quality, etc.) or drops in rankings (not necessarily because they made a site fall just by existing, but because their value might be removed)
- Finding links that point to 404'ing pages
- Determining the relative link popularity of a given page on a site compared to others
- Investigating the links to a domain or page we're considering getting a link from
- Finding links to competitors that outrank a client for potential acquisition
- Finding links that point with less than ideal anchor text that we can request modifications to
- Researching a site's internal link structure to find problems or opportunities
- Checking links between link partners or multiple sites owned by an entity
- Probably 100s more (seriously, I'm in there 3X a day or more when doing SEO stuff)
Some questions:
- How do you employ Yahoo! Site Explorer and Yahoo! link searches?
- What tasks do you complete with this data?
- What features are sorely missing (or poorly implemented)?
- What do you think Yahoo! gets out of Site Explorer? (Don't get me wrong, I love it; I just don't get the business case)
Since it's such a critical tool for me, I'm fascinated to know how others leverage the service.
Huge work done and a great link analysis round-up. Some tools that overcome the weaknesses listed:
1/ link diagnosis (mentioned by Will) : shows: anchor text, PR, nofollow
2/ SEO Link Analysis (FF extension) : shows anchor text, PR, nofollow;
3/ LinkHounds : sorts links by domain name (no multiple results from sitewide links)
4/ Domain Backlinks Checker : sorts results by TLD and also lists all pages indexed by Yahoo the link appears on.
5/ SEOquake (FF extension) to overcome "importance" issue : shows each linking page backlinks, PR, domain strenth, age, etc...
I for one use Yahoo! (and YSE) a few times a day for competitive research, link building and reputation management tasks.
Many people will say YSE is not accurate but that's the best tool we have so far.
Others argue that it is not wise to follow your competitors steps but I say, do it wisely, and besides you simply never know your market until you learn your competitors' strengths...
And thanks for the link to my post at SEJ btw!
Whoa, nice list. Thanks!
More tools, thanks Ann
I have been beat into the ground recently by a competitor. Doing a backlink analysis on them was one of the most revealing things I have ever done.
I have never seen any one SEO firm lean so heavily on one type of link. But it worked or is working. So now I know how they beat my pants off me. So now I know I need more resources in order to compete, its been fun and humbling.
Can you share more details? Sound interesting.
Sean
YSE is a tool that I pray Yahoo doesnt take away from us one day, as I too dont get the business sense of it. Maybe they can add some Google ads in there to compensate for the bandwidth *shhhh*.Most of the weaknesses you outlined can be remedied with Yoast's FF extension SEO Link Analysis. With the added info from the extension it furthers my desire for a TSV export to really drill down link strength in a table format.
yeah, it's a good addon for the yahoo site explorer - cheers for sharing. It has one big drawback in that it doesn't download all the data in the TSV: only the standard Y!SE data.
If it could download the enhanced data too, it would be a great addition.
Hey it's not like it's that easy to do that ;) maintaining this is already a *&* in the *&*( :)
I use your link analysis plug in and its great! Thanks for writing a great piece of software! :-)
I second that - a HUGE thank you Joost for the link analysis plugin, its indispensible for SEOs and it also does its magic in Google Webmaster Tools backlink data.
I'm going to check out Joost's plugin mentioned above, but I use link diagnosis which pulls data from Y! but then overcomes a lot of the weaknesses you mention (not all of them).
But yeah, I'm in there a lot and I imagine that the guys on the team here are in there even more...
Hi Rand,
InLinks lists form Yahoo Site Explorer have also another issue: they include also pages containing only an HOTLINK (and no backlink) to any image hosted within the domain you are investigating.
It's a big issue in some cases.
Anyway I love the possibility to access data through APIs and then extract information frrom it at my wish. This is Vital!
I wonder Rand, what your experience has been with the YSE API vs the YSE web results. They tend to give different numbers and results, and when I asked a Yahoo engineer about that, he said it was because of "more filtering happening when API results are given".
Do with that what you want :)
Hey Rand, I did not know you loved me (or Turkey) so much :-p.
Of all the country extensions, he uses the com.tr extension.
Huge spoiled, self loving and gloating smile behind this comment :-p
I love you too man. P.S. Thank you for helping us again with a well researched article.
Hi. Just to let you know that have added SEOMOZ.org to Spotlight Ideas Top 100 Advertising, Marketing, Media & PR blog list (with blog entries being placed into genres).
We use Yahoo! Site Explorer on a daily basis - it's an incredibly important tool for what we do. Thanks so much for posting this!
If you use firefox, this bookmarklet script automates the advenced query for getting accurate backlinks in yahoo site explorer.
javascript:location.href='http:/'+'/search.yahoo.com/search? p=linkdomain%3A'+document.domain.replace('www.','')+' -site%3a'+document.domain.replace('www.','')+'&n=100';
Another lazy way to use yahoo site explorer is to add the Firefox addon called yExplore.
You'll then be able to right click on the any webpage and choose yExplore from the context menu.
We Love Yahoo site explorer , my client has finished two to three URL Rewriting in their sites and forgot to do 301 redirection so before 1 week yahoo site explorer helped to find all the Old URLS of their's site.
so Y site explorer can help you to find these type issues or you can check your current back links which google is not showing.
hopefull, yahoo'll increase some feature which it needs to updates.
Eric
Excellent post Rand! Learned a lot of stuff, and its nice to know other people do the same types of queries. It helps to boost confidence, and f course build best practices in the SEO community.
Which is one thign I love about SEOmoz. Its a living vibrant communty!
Cheers,
-Bart
Good post Rand.
"Nofollow Included - in a tragic move, nofollow'd links are included in the list with follow'd links, and no differentiation exists between the two, forcing SEOs to do their own research page by page to find which links the engines might actually be counting."
This "weakness" can be solved quiet easily wist Joost de Valks Link Analysis FF extension which you can find under https://yoast.com/seo-tools/link-analysis/
With this extension you can immediately see if its a dofollow link, a nofollow link and the PR of the linking site.
Hope it helped.
Cheers
Ups, I saw somebody posted it already. Sorry didn't read through all comments.
"...as Google made their link query useless in 2003..."
I was still using pen and parchment in 2003. Can someone tell me how Google made their link query useless in 2003? Is the data it provides inaccurate or just incomplete?
Excellent post providing much to think about. Prior to reading SEOmoz on a daily basis we've been at a loss for good link buidling. Basically we had a site of paid sites and then worked our way down the list.
This Yahoo! organic approach seems to be the best way to go. Thanks again.
I feel like a bloody Jedi in training sometimes and Rand is like Yoda.
"Critical to measure, it is, quantify and review links. Hmmmmmm."
YSE is very important for us. Typically use it for backlink analysis for clients to illustrate/educate why backlinks are important and competitive research to try to snag the same links as the competition.
I've also found that it is updated very frequently. Witnessed a site that was published with very little fan fare and YSE was able to detect links to the site within a day or two of going live.
Great post. Any serious SEO should be using Yahoo to at least view link building progress.
webuildpages had a very good link research tool that is now a paid service. I have not reviewed the paid version and cannot comment on its current effectiveness.
What a great post. I have used that Yahoo tool in the past but didnt' really know how to get the most out of it. Once again Rand you have shown us the light. Keep up the good work.
Okay, to be honest I had not been using Yahoo tools that much mainly because as hismove eluded to, and their lack of a deal with MSN, I did not want to develop a dependence on tools I could not plan on having in the months or years ahead. Your post has brought up some interesting info so I have to rethink my position. @ moz pit...once again the comments after the excellent post only add to its value. Thanks for all your input.
This is a great read and super helpful. Thanks for the tips!
And I also think that another website of mine has now much less backlinks than it had two weeks ago?! I haven't noted the numbers so I can't say for sure but I'm pretty sure it had more inlinks than it has now...
Any thoughts? Thanks!
A few sites I have checked are doing the same thing. One lost 550, another lost 168 and another lost a few thousand. Any thought or explanation on why this happens?
Great post. It opened my eyes to different ways to use the tool. I've always used it to perform competative research but there are some other powerful uses that you have clearly identified. Thanks a bunch!
This is a good post from Rand. Obviously, since Open Site Explorer, there is little need for Yahoo's Site Explorer but the post provides some usefull ways to use the data that is available.
I'm puzzled.
I started monitoring a website where nothing was done in regards to SEO since 2006. They had 193 inlinks accoding to Yahoo Site Explorer on May 1st. Today is May 15th, and Yahoo Site Explorer only shows 125 inlinks (same settings/entire domain etc.).
How can this be?
Thanks to everybody who can help me understand what's going on there!
IMO, Yahoo site explorer and Google web master tools shows nearly same in bound links, the main thing is when you are doing link building for clients he won't check his webmaster tool account but he can surely check back links in Google for whatever stuffs you had done. As Google not showing all links pointing to website, its safe and advisable to check IBL's from yahoo site explorer. So you can come to know about your link building status, how much you are progressing, as well as client also come to know about. However it shows Nofollow links also, but it won't matter if you are engaged with dofollow link building.
Thanks.
I use Yahoo's links information several times a day. It's not 100% accurate, but it's the best free tool right now. I have used Joost' plugin, it gives great information: anchor text, not found links, nofollow links, unfortunatelly it stopped working a while ago.
I use YBL as typical CSI in my SEO projects. Still, my problem is, that different tools and queries report different numbers of backlinks. Only 2, Mikes Marketing Tools and SEO Centro report right number, out of 20 seo tool sites, I have tried.
Jaak, https://seoapplied.blogspot.com
YSE is used here in conjuction with the SearchStatus FF plugin for most of the reasons mentioned above:
initial keyword research
link building research
- researching potential sites to get links from
- monitering current internal and externals
looking for scrapers pulling our content
error pages etc.
I also use it for monitering content grouped by them within our pages since we publish so much on daily basis. This helps maintain our IA with a light pyramid style approach.
Great post Rand, def. one I'm going to pass along to some other SEO peeps.
Cheers,
@trontastic
Rand, this was a great post. I definitely don't use Yahoo as much as I should for my link work, but much of that stems from the fact that the extensive link data is a blessing and a curse. Mostly because of how many links you have to comb through to find the really valuable ones.
However, after reading this I think I might give it a little more love than I have in the past. You shared some great information that I think I may be able to use to my advantage.
I use it to find the number of inbound links, and honestly that was about it, until now. After reading this I have a better idea of what it can be used for, and I can see how powerful this tool is. Thanks for the info!
BTW, there are a few letters transposed in your post. woudl & informatino. :)
I should ammend my vote. I originally voted for "it's valuable, I use it a few times a week" but actually I use Yahoo's info on a daily basis, I just do it through the SEO for Firefox plugin to check link numbers. So I may not always go to yahoo.com for the data but I am constanly using the data that they provide.
We use Y!SE very often and will check out the plugins (which I have known about but been too lazy to try) mentioned.
I think Y! gets additional search traffic out of offering Site Explorer. I know I always use my FF3 toolbar search box with the drop-down selector for choosing between the various engines, and whenever I use Y!SE, the regular search I do following a Y!SE query is always though Yahoo! when it would normally have been through Google.
Edit: I know this doesn't account for a lot of traffic, but it's valuable SEO/SEM traffic in many cases.
I use Yahoo Site Explorer multiple times a day. I find that a combination of tactics works best, including using:
- advanced link operators
- link diagnosis
- Joost's PR and Anchor text tool that integrates into YSE
I don't know what I'd do without it!
I would be very interested in knowing thoughts on SiteExplorer versus SiteExplorer API - do Yahoo! fudge the data in their API like Google?
very interesting. I have a Yahoo API, but have not noticed any difference... but I mainly use it for rankings.
It would be a nice thing to actually test each and every query... would take some work and some automated tools so as to get data samples close together elminating time as a variable.
But it sounds like fun.
Very helpful post...I'll definately start looking at it more now i can see the full potential and opportunities.
By the way - can someone explain something to me please. When you get the list of link results on YSE you can click an option on the top right called "Result Details"...if you click the multiple lined image option it adds a line to each results saying "Authenticate the site to see additional information". You need a Yahoo Account to do this so what does it actually do and what extra does it give you?
I'll certainly use Yahoo data more frequently now. Especially as I have noticed for the first time this month, that Google seems to have blocked the scraping of WebPosition reports.
Ok last week I noticed that Google has blocked Web Position as well...I wonder why they have done this and if there is another program or way around it??
I notice sometimes that an external backlink found ok to our site in YSE shows up as 'Link not found' in Google Webmaster Tools. It would be interesting to know why these differences occurr for the same backlinks.
I'm still not sure why these display different results like this - when I check the backlink to us they do indeed still exist and seem correctly coded. Google acknowledges it was found in the past but can no longer find it, however YSE still finds it.
Mysterious..!
I like using it to try getting a glimpse of what real traffic volume might look like for a particular niche keyword. IF a site ranks at the top for a keyword AND you can see in yahoo that it links back to one of those free tracking sites, (link:freestatssite.com keyword) you can view the site's stats and see what kind of hits it's getting for a target keyword. It's hit or miss but helpful when it works.
Yes good post but the biggest problem is that Site Explorer is limited to a 1000 links
What will you guys use when Yahoo disables this tool? Anyone aware of any crawlers being built that we could use?
Another valuable post, Rand. Good stuff.
I'd love YSE a lot more if it were more reliable. I get errored out frequently when looking for link data.
It's an extremely robust tool and we rely on it in doing our initial keyword and link building analyses. I definitely think that as I grow into my role a bit more, it's something I'll be using more and more in proportion.
I might be wrong about this but try signing into yahoo. I seem to remember reading somewhere that you get less errors when you're signed in (and certainly I don't recall getting any errors recently and I always use it signed in)
I do notice that Yahoo does give more comprehensive results than Google. But the Google webmaster tool shows links that Yahoo does not. Its a must to inegrate both into your daily SEO routine.
i've been using it for doing anti-spam patrol on websites we have with lots of scrapers..
=)
I have this recurring nightmare where I go to do link research and Yahoo site explorer doesn't exist anymore.
Seriously though... what would happen if they decided to pull this service similar to Google or MSN?
I'm one of the 5-10x/day kinds of users, and it makes me nervous that I have to rely so heavily on the tool.
What would YOU do if it suddenly disappeared?
I find Site Explorer really useful.
I use it alot to track how my linking campaigns are going - see what sites have included my links, and using the rough guide of 'most important first' I can get an idea of what quality these links are providing.
I've also found quite a few problems with it screwing up lately, and have previously experienced problems when downloading TSV files.
Nice post - We too use YSE about 3/4 times daily (if not more). We use it for link acquisition as well as to determine the amount of juice being passed through to a domain or a particular page, as you said Rand a type of page strength as opposed to PR.
We also use it for a whole host of other stuff, but to be honest you pretty much covered it all... competitor analysis, monitoring site growth/successful linkbait campaigns etc. We also like all the modifiers.
Good question on why it exists... I suppose it gives Yahoo! something Google does not provide. Maybe more of a reason for us SEOs to use Yahoo! more?
Rand, it's posts like this and the Give it Up post that have me going back to Y!SE more and more. As a noob in search of learning new techniques, what you've mentioned here and in other posts have helped me in finding sites for link opportunities, assessing link value and developing plans.
Previously I'd only ever used Y!SE to look at links to sites without much use of advanced operators. It's now a whole new world.
Thanks for sharing the knowledge.
This is a very excellent post. I learned something valuable.
Thanks!
FROM RAND: I removed the link here - we ask that folks don't link drop without relevance.