We've updated the SEOmoz API! Our new Free API gives you Domain Authority, Page Authority, Anchor Texts, mozRank, and also up to 500 links. You can get all your links and metrics with our paid API, the Site Intelligence API.
With Love,
SEOmoz
******
We've been pouring in hours on Linkscape, growing the index (46+ million domains & 36+ billion pages), making the updates closer together and refining the quality of our metrics. Today, we've got two big announcements that should be exciting for nearly everyone in the SEO development and analysis space. First off, we're launching a free Linkscape API to access two of Linkscape's most popular metrics - mozRank of a URL and the number of external links to a URL. Second, we've launched a list of the top 500 sites & pages on the web, ordered by the number of unique root domains linking to them (alongside other interesting SEO metrics).
The Free API
Since Linkscape launched in last October, we've had a lot of feature requests from all kinds of people: big-time agency SEOs, freelance SEOs, tool vendors, friends, and competitors. One thing that really hit home with us, from all of these people, was the large number of requests for a Linkscape API. Many of you have been scraping Y!SE, Google, and other data vendors to plug into internal tools and your public widgets for a long time. We've done our fair share of that kind of work, and we know what a pain it can be.
So to answer that call we are unveiling the completely FREE mozRank API. We've put together a lot of resources to get you started:
- API Wiki - 99% of the API documentation is here. It includes a few key pages:
- Getting started - a good place for newcomers to the API to follow the ordered steps to get going with the service.
- Generating a valid request - a good breakdown of the request URL you'll send the API.
- Request/Response - Page detailing request response fields.
- mozRank API - Page detailing the mozRank API call and response.
- Google Group - Mailing list for users and developers of the API to ask questions, get help, and communicate with each other.
- api [at] seomoz [dot] org - You can also email us direct for API questions/help/sales.
So what's in the API and why should you care? First, we're offering mozRank, our logarithmic scale metric of global link popularity based on the importance of links pointing to a page. It's similar in intuition to the algorithms powering Google's PageRank, Yahoo!'s WebRank & Microsoft's StaticRank. Second, we provide a raw count of external, juice-passing links. This number indicates how many links we've seen across the web pointing to a given page from external sites, excluding no-followed links.
We chose those metrics to share thanks in part the results of correlation testing we've been doing for a long time (this research should launch on the blog later this week). We've traditionally looked at metrics like Google Toolbar PageRank for SEO and internet marketing purposes. The consensus we've heard is that this metric isn't great for predicting rankings, but it's all we've got. We're hoping to change that.
Our results show that this instinct about Toolbar PageRank can be statistically supported. It's not completely wrong, but I think you'll agree that this metric is probably not designed for some of the marketing purposes to which we are putting it. In particular we've found that as Toolbar PageRank ages, it drifts out of correlation with rankings. With updates coming as infrequently as six months to a year, this effect can be substantial. Toolbar PageRank also seems to be correlated with a variety of effects, from link popularity, to domain authority, to penalties and trust. This makes many uses quite difficult: did your Toolbar PageRank drop because you lost some important links? Or were you caught with paid links? Is the new juice you seem to have because of that linkbait post you wrote, or because your domain has achieved the next level of global authority in general?
We believe that mozRank is a much better metric for these marketing and optimization purposes (and is often exceptionally valuable when compared up against Toolbar PageRank). We are committed to keeping mozRank as updated as we possibly can: so far we've updated mozRank twice, with one to two months between updates. We're actively working to make the updates closer to one month apart. MozRank is also measuring a single effect: link popularity. Coupled with the count of external links (also included in the free API), you can get a fair measurement of two of the most important ranking dimensions. We're going to continue measuring the other effects I mentioned (domain authority in domain mozRank and trust in mozTrust), along with many other factors in our web-based Linkscape tool.
BTW - You can already see the mozRank API in action with Quirk's SearchStatus toolbar:
Along with the SEOmoz Toolbar (which shows a lot of additional metrics for PRO members), this is one of only two toolbars that currently show mozRank & external link counts.
The Top 500 Most Linked-To Domains & Pages on the Web
We've also compiled a list of the most linked-to sites & pages on the web, according to our index. We found this data fascinating - seeing which domains and URLs had amassed incredibly high numbers of links, looking at the stats surrounding them and parsing the cause for their meteoric link counts. We also made some excellent discoveries about spam filtering, the value of our metrics and the power of having a plug-in or embedded widget that links back (just look at how many of the top linked-to pages & domains are popular as a result of this phenomenon).
Since the chart mostly speaks for itself, I'll hold off on a thorough analysis, but needless to say, there are a great number of interesting observations and hypotheses to be drawn from the list. And, while it's awesome to look at and play with now (there's a downloadable CSV as well), it's going to get even more exciting as we update the rises and falls as Linkscape updates and the web progresses. Just as an example, if we had run this on our first index in October of 2008, I doubt we'd see Twitter near the top. Today, it's in position #69, and I'm guessing it's trending upwards in the next crawl.
We hope you enjoy both of these new features around Linkscape. Spread them around and report your findings and applications.
Some interesting things I spotted from the list:
According to Linkscape, Quantcast has the 352th most important page on the internet https://www.quantcast.com/p-fcYWUmj5YbYKM. The reason for it getting so many links is because a version of the "Quatnified" analytics tag includes an anchor which includes the account id. This particular account id seems to be for Typepad Network (the blogging platform).
By checking backlinks in Yahoo you can see it is linked to from many different different domains (not just subdomains of typepad.com), but clearly from the PageRank shown in the chart (Gray) these links are not being counted. A search for "Typepad Network" on Google confirms that the page doesn't have the power to rank even for the title tag of the page. This must be a manual linkjuice block from Google - I wonder if Linkscape will need to take into account these type of links when attributing mozRank.
It was also interesting to find TinyURL as the 88th most important website and yet it's homepage didn't even feature in the 500 most important pages. Has this list taken into account 301 redirect as most URLs to TinyURL will be of this form?
You're getting at some good issues that we've also been thinking about.
In some ways, and in some cases we are taking a "naive" approach to link juice right now, in so far as this kind of stuff can be naive ;) What you suggest about the Quantcast page sounds similar to other things we've seen. We're working on integrating some more advanced techniques, but so far we don't see this very often and only for obviously degenerate cases like this one.
We do count up links to pages that just 301 away, which is probably why TinyURL shows up. I'd argue this still means that TinyURL is deeply embedded in the fabric of the web. They may not have much original, rankable content on their site, but they have a great opportunity. Those might not be as good as links to non-301 redirect pages, but users see TinyURL links every day. And they click on them! Clearly TinyURL has earned some authority and trust among users.
Hi Nick... I'm often unsure about the approach Linkscape takes for 301s. I can see the uses that counting links to URLs that 301 seperately from the target URL but I also think there are as many, if not more uses, of combining the 301 URL with the target URL (as this is essentially what humans and search engines see). I haven't found a way in Linkscape to see a combined picture - maybe I'm missing something - but if not this is definitely something you should consider.
This is an issue we struggle with actually. We've thought about doing what you're suggesting a lot. Having your feedback on the issue is much appreciated :)
I'm resisting the urge to explore the top 500 all day. There are a number of domains I've never visited near the bottom of the list. This feels like a modernized Fortune500. Nice work team!
I agree. The top seems pretty obvious, but the bottom of the list is really interesting.
For those interested, I wrote a quick PHP wrapper class for the Linkscape API, available here:Linkscape API PHP Class
It should make interacting with the API a bit easier. If anyone has any improvements to my implementation please let me know! When I have some time tomorrow I'll add some demos and a CakePHP component.
Thanks for the API - this is very cool stuff. Looking forward to the full rollout!
Your code looks good. However, in the era of PHP5 I would suggest you to stop writing PHP4 code and upgrade your class to PHP5.
I've bookmarked the URL for later retrieval when I get to seriously looking at this API!
I went ahead and changed the constructor to use PHP5's __construct method, good call. I'm assuming thats what you meant by "PHP4" code - any other suggestions?
Your code looks good. However, in the era of PHP4 I would suggest that you stop writing PHP5 code and downgrade your class to PHP4.I've bookmared the URL for later retrieval when I get to seriously looking at this API.
Sweet :)
We'll take a peek at it. Maybe we can post it on the wiki under sample applications? (with attribution of course)
Cool stuff. I checked out the demo code, and since there wasn't an example for .Net....well, there is now.
https://gathadams.com/2009/02/10/seomoz-api-wrapper-in-vbnet/
Sweet, thanks for that. Do you mind if we add the code to the sample apps page in the Wiki? We'll be happy to give you credit.
No problem.
Really? Exciting News about Moz API.. keep it up.. Sharanyan
Yes, the long awaited API. Can't wait to play around with with. Still waiting for my cupcake though.
Fantastic work guys - and well done for making it open! I've not played around with it yet but I'm sure I will be.
Major shout out to Quirk for including this API in their new SearchStatus bar.
Craig at Qurik has been working with us on getting mozRank into the SearchStatus bar for a while. We were glad to have it included! It's got a bunch of other metrics and tools built in too.
Check it out! (and enable mozrank from the right-click menu on the "q")
Wow, definitely some exciting news. Previously, I thought I saw that SEOmoz was renting the API at a $5000 monthly cost? Am I mistaken or was that another service?
We do have a full API that provides much more access to all our metrics and will also contain access to pull link lists (like the advanced reports do now). That's still in development, but yes - the costs will probably start somewhere in the $5K / month range.
$5k/month!? I suppose it's a reasonable price for access to such valuable data, but still, too bad, as it puts it out of reach for most of us.
Of course, being in development, we're also developing (have not set) the right pricing.
Looking at the top sites, I see that macromedia.com is listed at #13. It caught my eye since I worked there until Adobe bought them 05. Obviously macromedia.com now redirects to adobe.com and has for some time. So why would the links be counting towards macromedia and not all counting towards adobe instead? I thought if a site was 301 redirected that the links would then count towards the site being redirect to.
NM: (Read Nick’s comment below. He knows more than me.)
Because there still are so many live links aimed at macromedia.com we included it in the list.
In our index we credit the target 301s (adobe.com) with all of the juice from those links, but we can still report about the link profile of the source (macromedia.com) separately.
We think it's pretty handy to be able to see those separately (e.g. gauging the value of the 301 vs having it be a separate site).
someone's gone a little dog gone down thumb crazy on this thread.
bitter anyone?
I thought it was a good question.
i was referring to every comment above mine that has at least 1 thumb down too. a bit curious.
oh yes, I see what you mean. My super secret psychic powers tell me that there IS something funny afoot. Let's just pay those down thumbs no mind...
Rand,
Givin' it all away again.
Your the man.
How often will the Top 500 pages and Top 500 Domains pages be updated? Are these stats in real time or are they gathered and published only periodically? Or updated only when linkscape is updated?
We'll update the list when we update the index. So far that's been about once every month or two. But we've only had two updates since launch. We're working hard to bring that closer to one month.
I like your name :)
I want to strongly encourage anyone using the API to join our google group. I'm not saying you have to share your secrets (unless you want to!), but it would be great to hear about any issues or surprises you're running into.
I'll second that, Nick. It looks like the Google Group discussion is well underway. I set up the daily email notifications which makes it easy to keep up with the discussion in a passive way.
Thanks for the update and API. Looking forward to checking it out this upcoming week!
I'm sure I saw a link to th api in the right rail the other day, which had me intrigued that something was about to be launched, this is really exciting! Congratulations on the launch and Top500 data sheet. I will be reading up on the wiki. :)
Very nice step. It's nice to have Linkscape and MajesticSEO so now we need to rely less on the same ole Yahoo Search. But there's still some ambiguity.
Like checking https://www.Sciencedaily.com, Yahoo says this page has 346,000 links (excluding internal links). MajesticSEO says this page has been linked to from 30,450 Domains (probably means more links overall).
But Linkscape says:
Links to the page sciencedaily.com --- 1,043 linksNow the catch seems to be in the phrase:
"not counting links from sciencedaily.com or links that don’t increase the mozRank".
So I think the reason for the big difference can only be the links that LinkScape is discounting as they don't increase mozRank (or am I missing something?).
Now I do respect your knowledge and expertise Rand, and it's pretty obvious you have earned quite a reputation in the IM world. But as it's evident, even the SEOMoz Search Ranking Factors survey showed top SEOs rarely agree on one point. So many might not agree on your calculation of what domains/links pass Pagerank or Link Juice as seen by Google.
I know you've mentioned mozRank rather than Pagerank, but most of the SEO world is trying to optimize pretty much for Google.
In the light of all this, is it possible that you somewhere disclose what type of links the Linkscape discounts?
Linkscape's link counts include only followed links, so nofollows won't appear in those numbers. We also treat external and internal links differently, so when you see an external link count, that means only those links off the domain that point to that page.
We're still about 1/3 to 1/5 the size of Yahoo! or Google, but Majestic is a bit more troubling. Since they don't crawl the web like an engine, but instead take data back from surfers using their distributed app, they see a lot of URLs the engines probably never will. Their data is also amalgamated from many years, so in our look at freshness, we found that fewer than half the links they reported still existed in a rough sampling.
Also - from Linkscape's data, I see:
Not sure where your 1,043 number comes from - you might have checked https://sciencedaily.com (without the www), which redirects.
I'm usually the overly-cautious super critical one at the mozPlex, but I'd like to think we may be in better shape than Rand suggests in general. We've done a couple of studies that suggest that even if you take into account some of the freshness issues (which are monstrous to deal with, I sympathize with the issues Majestic has), I think we may be closer to 1/2 in general and often very comparable.
If there's community interest, I should probably do a post about how we're measuring the quality of the index: where we're in good shape and our goals to go out of beta.
Oh yeah, sorry about that. I went to Linkscape from the Top 500 page (where Sciencedaily.com is) and pasted that directly without the 'www'.
About Linkscape being 1/3 the size, that clears more of the picture with respect to yahoo. I checked 3 large sites and Yahoo's figures are around 3-3.5x those of Linkscape.
Cool. Congrats on the launch! Just been playing with linkscape a bit over the past few days. Some interesting data in there :)
Thanks for doing this.
How about adding the rank of the domain to a column in the linkscape advanced report for incoming links? (provided you track beyond 500) I think it'd be informative to know if we have some top 1000 incoming links.
This is very cool.
One suggestion though - https://apiwiki.seomoz.org/Available-API-Calls - it's not entirely clear which calls free members can make?
How can I make calls to get the number of links to a URL? The wiki suggests that only mozrank is available. Am I missing something very obvious?
Well done, BTW, on releasing this.
Rgds
Richard
From the first paragraph of that wiki page:
Hopefully that answers your question!
You can get the number of external, juice-passing links in the mozRank API (which is the free one we're releasing today)
Huzzah!
do people really need all these fancy linking tools or can they just make great web pages and write great content? im not sure.
Making SEOmoz data programmatically accessible is part of a larger industry tend towards quantitative statistics and verifiable facts - which I think is a great thing. Hard data is a tool honest consultants can use to beat down people full of hooey!
go ahead and downvote me. but it just proves that i struck a nerve, for one reason or another.
I think you make an excellent point. Great content is extremely valuable, and often overlooked by many web masters and site owners. It's difficult to rank without great content, and easy to become competitive with it.
I think our community is pointing out that in many areas there's a lot of excellent content and ranking there is extremely competitive. Measuring progress toward your goals with things like traffic analytics is very natural. And so is counting links (and other derivate measures), for the same reasons.
I hope both you and our community can see the overlap and middle ground here.