Here at Moz, we take metrics and analytics seriously and work hard to ensure that our metrics are first rate. Among our most important link metrics are Page Authority and Domain Authority. Accordingly, we have been working to improve these so that they more accurately reflect a given page or domain's ability to rank in search results. This blog entry provides an overview of these metrics and introduces our new Authority models with a deep technical dive.
What are Page and Domain Authority?
Page and Domain Authority are machine learning ranking models that predict the likelihood of a single page or domain to rank in search results, regardless of page content. Their input is the 41 link metrics available in our Linkscape URL Metrics API call and their output is a score on a scale from 1 to 100. They are keyword agnostic because they do not use any information about the page content.
Why are Page and Domain Authority being updated?
Since these models predict search engine position, it is important to update them periodically to capture changes in the search engines' ranking algorithms. In addition, this update includes some changes to the underlying models resulting in increased accuracy. Our favorite measure of accuracy is the mean Spearman Correlation over a collection of SERPs. The next chart compares the correlations on several previous indices and the next index release (Index 47).
.gif)
The new model out performs the old model on the same data using the top 30 search results, and performs better if more results are used (top 50). Note that these are out of sample predictions.
When will the models change? Will this affect my scores?
The models will be updated when we roll out the next Linkscape index update, sometime during the week of November 28. Your scores will likely change a little, and may potentially change by as many as 20 points or more. I'll present some data later in this post that shows most PRO and Free Trial members with campaigns will see a slight increase in their Page Authority.
What does this mean if I use Page Authority and Domain Authority data?
First, the metrics will be better at predicting search position, and Page Authority will remain the single highest correlated metric with search position that we have seen (including mozRank and the other 100+ metrics we examined in our Search Engine Ranking Factors study). However, since we don't yet have a good web spam scoring system, sites that manipulate search engines will slip by us (and look like an outlier), so a human review is still wise.
Before presenting some details of the models, I'd like to illustrate what we mean by a "machine learning ranking model." The table below shows the top 26 results for the keyword "pumpkin recipes" with a few of our Linkscape metrics (Google-US search engine; this is from an older data set and older index, but serves as a good illustration).
As you can see, there is quite a spread among the different metrics illustrated, with some of the pages having a few links and others 1,000+ links. The Linking Root Domains are also spread from only 46 Linking Root Domains to 200,000+. The Page Authority model takes these link metrics as input (plus 36 other link metrics not shown) and predicts the SERP ordering. Since it only takes into account link metrics (and explicitly ignores any page or keyword content), but search engines take many ranking factors into consideration, the model cannot be 100% accurate. Indeed, in this SERP, the top result benefits from an exact domain match to the keyword and helps explain its #1 position despite its relatively low link metrics. However, since Page Authority only takes link metrics as input, it is a single aggregate score that explains how likely a page is to rank in search based only on links. Domain Authority is similar for domain wide ranking. The models are trained on a large collection of Google-US SERP results.
Despite restricting to only link metrics, the new Page and Domain Authority models do a good job of predicting SERP ordering and improve substantially over the existing models. This increased accuracy is due in part to the new model's ability to better separate pages with moderate Page Authority values into higher and lower scores.
.png)
This chart shows the distribution of the Page Authority values for the new and old models over a data set generated from 10,000+ SERPs that includes 200,000+ unique pages (similar to the one used in our Search Engine Ranking Factors). As you can see, the new model has "fatter tails" and moves some of the pages with moderate scores to higher and lower values resulting in better discriminating power. The average Page Authority for both sets is about the same, but the new model has a higher standard deviation, consistent with a larger spread. In addition to the smaller SERP data set, this larger spread is also present in our entire 40+ billion page index (plotted with the logarithm of page/domain count to see the details in the tails):
.png)
One interesting comparison is the change in Page Authority for the domains, subdomains and sub-folders PRO and Free Trial members are tracking in our campaign based tools.
.png)
The top left panel in the chart shows that the new model shifts the distribution of Page Authority for the active domains, subdomains and sub-folders to the right. The distribution of the change in Page Authority is included in the top right panel, and shows that most of the campaigns have a small increase in their scores (average increase is 3.7), with some sites increasing by 20 points or more. A scatter plot of the individual campaign changes is illustrated in the bottom panel, and shows that 82% of the active domains, subdomains and sub-folders will see an increase in their Page Authority (these are the dots above the gray line). It should be noted that these comparisons are based solely on changes in the model, and any additional links that these campaigns have acquired since the last index update will act to increase the scores (and conversely, any links that have been dropped will act to decrease scores).
The remainder of this post provides more detail about these metrics. To sum up this first part, the models underlying the Page and Domain Authority metrics will be updated with the next Linkscape index update. This will improve their ability to predict search position, due in part to the new model's better ability to separate pages based on their link profiles. Page Authority will remain the single highest correlated metric with search position that we have seen.
The rest of the post provides a deeper look at these models, and a lot of what follows is quite technical. Fortunately, none of this information is needed to actually use these Authority scores (just as understanding the details of Google's search algorithm is not necessary to use it). However, if you are curious about some of the details then read on.
The previous discussion has centered around distributions of Page Authority across a set of pages. To gain a better understanding of the models' characteristics, we need to explore its behavior on the inputs. However, the inputs are a 41 dimensional space and it's impossible (for me at least!) to visualize anything in 41 dimensions. As an alternative, we can attempt to reduce the dimensionality to something more manageable. The intuition here is that pages that have a lot of links probably have a lot of external links, followed links, a high mozRank, etc. Domains that have a lot of linking root domains probably have a lot of linking IPs, linking subdomains, a high domain mozRank, etc. One approach we could take is simply to select a subset of metrics (like the table in the "pumpkin recipes" SERP above) and examine those. However, this throws away the information from the other metrics and will inherently be more noisy than something that uses all of them. Principal Component Analysis (PCA) is an alternate approach that uses all of the data. Before diving into the PCA decomposition of the data, I'll take a step back and explain what PCA is with an example.
Principal Component Analysis is a technique that reduces dimensionality by projecting the data onto Principal Components (PC) that explain most of the variability in the original data. This figure illustrates PCA on a small two dimensional data set:
.png)
This sample data looks roughly like an ellipse. PCA computes two principal components illustrated by the red lines and labeled in the graph that roughly align with the axes of the ellipse. One representation of the data is the familiar (x, y) coordinates. A second, equivalent representation is the projection of this data onto the principal components illustrated by the labeled points. Take the upper point (7.5, 6). Given these two values, it's hard to determine where it is in the ellipse. However, if we project it onto the PCs we get (4.5, 1.2) which tells us that it is far to the right of the center along the main axis (the 4.5 value) and a little up along the second axis (the 1.2 value).
We can do the same thing with the link metrics, only instead of using two inputs we use all 41 inputs. After doing so, something remarkable happens:
Two principal components naturally emerge that collectively explain 88% of the covariance in the original data! Put another way, almost all of the data lies in some sort of strange ellipse in our 41 dimensional space. Moreover, these PCs have a very natural link to our intuition. The first PC, which I'll call the Domain/Subdomain PC projects strongly onto the domain and subdomain related metrics (upper panel, blue and red lines), and has a very small projection onto the page metric (upper panel green lines). The second PC has the opposite property and projects strongly onto page related metrics with a small projection onto Domain/Subdomain metrics.
Don't worry if you didn't follow all of that technical mumbo jumbo in the last few paragraphs. Here's the key point: instead of talking about number of links, followed external links to domains, linking root domains, etc. we can instead talk about just two things - an aggregate domain/subdomain link metric and an aggregate page link metric and recover most of the information in the original 41 metrics.
Armed with this new knowledge, we can revisit the 10K SERP data and analyze it in with these aggregate metrics.
This chart shows the joint distribution of the 10K SERP data projected onto these PCs, along with the marginal distribution of each on the top and right hand side. At the bottom left side of the chart are pages with low values for each PC signifying that the page doesn't have many links and they are on domains without many links. There aren't many of these in the SERP data since these are unlikely to rank in search results. In the upper right are heavily linked to pages on heavily linked to domains, the most popular pages on the internet. Again, there aren't many of these pages in the SERP data because there aren't many of them on the internet (e.g. twitter.com, google.com, etc.) Interestingly, most of the SERP data falls into one of two distinct clusters. By examining the follow figure we can identify these clusters:
This chart shows the average folder depth of each search result, where folder depth is defined as the number of slashes (/) after the home page (with 1 defined to be the home page). By comparing with the previous chart, we can identify the two distinct clusters as home pages and pages deep on heavily linked to domains.
To circle back to search position, we can plot the average search position:
We see a general trend toward higher search position as the aggregate page and domain metrics increase. This data set only collected the top 30 results for each keyword, so values of average search position greater than 16 are in the bottom half of our data set. Finally, we can visually confirm that our Page and Domain Authority models capture this behavior and gain further insight into the new vs old model differences:
.png)
This is a dense figure, but here are the most important pieces. First, Page Authority captures the overall behavior seen in the Average Search position plot, with higher scores for pages that rank higher and lower scores for pages that rank lower (top left). Second, comparing the old vs new models, we see that the new model predicts higher scores for the most heavily linked to pages and lower scores for the least heavily linked to pages, consistent with our previous observation that the new model does a better job discriminating among pages.
I use these numbers a TON, so this is great news! Nice work ladies and gents. It's really interesting to read about how you arrive at the numbers that you do.
It struck me reading through the post that everything is done with Google in mind. It makes sense, of course. Poor little Bing :)
We've considered building a separate metric for Bing at times, but we actually see that PA/DA, when measured in Bing's rankings, perform equally well and sometimes better than in Google, suggesting that the algorithms, at least from a link valuation perspective, are quite similar.
Do you think, since the metrics correlate with Bing better in cases, and the shortcoming of some Moz metrics is advanced spam analysis, that it's at least a sign that Google is a bit better at link spam analysis? (no surprise there really)
I also think it is interesting that they're quite similar, because it seems like there is less and less developments being made in link-based algorithms by search engines. This seems to be reflected in patents as well. A lot more of the stuff that seems to be changing seem to be related to content analysis, usage data, machine learning and social metrics. A lot of advancements in scaling, indexation, and faster processing.
Does SEOmoz have any plans to start working towards link spam analysis and other types of link analysis, like semantics, phrases, visual segmentation and content? I'd assume there is still a lot of work to be done with canonicalization as well. You guys seems much further ahead of Majestic in this, but I'd imagine it's an interesting problem to solve, and likely process intensive.
I'd love to see more of the type of metrics and practices in this post built into SEOmoz:
https://www.seomoz.org/blog/understanding-link-based-spam-analysis-techniques
Well, you know, we did ask this one guy for help, but he was busy :-)
Seriously, though, yes, I'd agree that Bing's spam analysis and link algorithms in general are probably less sophisticated than Google's, and that's likely what we're seeing with the correlation numbers doing well in Bing (it's also the case that raw metrics like # of linking root domains tend to have much higher correlations w/ Bing than Google, too).
As far as engines not focusing on link analysis... I'm not so sure. Part of me says they just don't write about it much. I really hope they're not ignoring it, because there's a long way to go on the spam analysis front.
And yes - next year, probably in the spring (maybe summer), we'll ship some measure of spam/manipulation for pages/sites. Before that, we're trying to get the index larger, fresher and focus on improving some of our existing metrics/data. We've got an internal goal of 100 billion pages across ~120 million domains in an index every 3-4 weeks (that might change depending on what we find as we extend our crawling).
I learnt more from the second half of this post than I have from anything I've read on the internet for quite some time. Thanks for the detailed insights... Fascinating.
No offend or something, this just runs through my mind upon reading the whole blog. My stupid mind is thinking of something like, why are you putting to much effort with the search result, but you don't have your own search engines? sorry for the words. But wouldn't it be great if you build your own search engines since you the criteria of search engine and how it works and i bet you have the greatest SEO team out there.
We considered it, but only briefly. The work required to make a full search engine is fairly intensive, and we'd spend a lot of cycles doing work that we suspect wouldn't actually bring value to marketers or our customers (and that's what we want to focus on).
Better prediction of any single page’s ranking results, principal component analysis, fatter tails—all good stuff. However, since any particular competitor in my clients’ niche may be using spam techniques and since you “don't yet have a good web spam scoring system” and “sites that manipulate search engines will slip by [you]”, as you say, how do I know if the mozRank, mozTrust, juice-passing scores, etc., that are used to calculate page authority and domain authority scores of a particular competitor have actually taken into account the spam factors of the linking pages?
It seems like I keep banging my head up against a wall because I want to trust OSE results but when I look at a page that may be spam and then look at OSE results for it that indicate otherwise, I wonder to myself if OSE has taken into account this factor and/or that factor in its analysis. And since I find it hard to be sure of the results for any particular page, I end up questioning the scores for almost every page.
Does what I’m saying make any sense? I mean, I’ve been using OSE and linkscape since day-one but perhaps I missed some important background information on how to interpret spam in OSE that has been covered in the past. Maybe I’m thinking OSE is something that it’s not. Maybe thick-headedness is putting blinders on me. Can someone help?
While I’m kind of on the topic, I often wonder about who’s using your paid API and what they are doing with it. It would be nice if there was some sort of central location that lists the companies that are using it to make public tools (paid or otherwise).
Agree, would be cool to get kind of "SEOmoz partners" page to see what great tools they did create.
Hi Metapilot - I'll try to explain as best as I can (and Matt may jump in here with more detail later).
At the current time, I'm not aware of any metric on the web publicly that measures spam/manipulation. Alexa Rank, Majestic link counts, the old Yahoo! Site Explorer link numbers, Google's PageRank, Blekko's data, etc. I believe this is because webspam, particularly link-based webspam, is very difficult to catch algorithmically without accidentally targeting a lot of non-spam, too. This is likely also the reason Google's team faces so many challenges in this arena (and why we all see a frustrating amount of link spam working).
That said, there is the mozTrust metric, designed to calculate the "distance" on the link graph any particular page/site has from a trusted seed set of sites. It's by no means perfect, but it can be somewhat helpful (unless the spammers have managed to get links from highly trusted sites, which sometimes does happen, just look at all the .edu results in "buy viagra" type queries).
When we calculate PA/DA, they use machine learning to get a "best fit" against Google's rankings. This means we include metrics like mozTrust and Domain mozTrust, as well as everything else in our repertoire (40+ metrics and the many hundreds of derivatives of those metrics) to get a picture of link-influence. This will, by no means, be a perfect way to evaluate sites and pages for their spamminess or lack of spam, but it's the best estimation I'm aware of about predicting performance.
In terms of API partners - yes! Andrew Dumont is actually working on something like that for launch in the next couple months.
Thanks for your reply Rand.
Brilliant, really interesting to see how PA/DA is evolving as a metric! I always feel a small amount of excitement when there's a Linkscape update XD
Do you guys think that you'll make a different set of tools or metrics for Bing?
I wish my brain could take stuff like this in easier, I'm going to have to re-read it several times I think... nothing to do with how it's written as it's written well, it's just my head and its dataphobia.
Will there be a video on this? I can soak it in better if so :)
*Note: There's a spam comment above me by "Karen Millen Dresses" lol
Great news, I use page authority and domain authority metrics on a daily basis so this update is greatly appreciated.When we calculate PA/DA, they use machine learning to get a Perfect fit against Google's rankings. I 've read all post and really like this post Thanks ............
Hi Matt, I have to confess I only read the 1st part of the post - but this one was perfectly well explained for a person with rather no technical knowledge like me.
Great information!
Awesome post, metrics and illustrations.
top info, guys! I support Justin Briggs wish for more link spam analysis and detection tools, as well as guidance and recommendations to deal with it...
Has anyone reproduced the Domain Authority / Page Authority, distribution full index charts since 2011? I would love to see how this data has changed in the past few years.
After reading this, the first thing I did was check out my own sites. Not sure what to make of it to be honest, not sure how these metrics play when you factor Panda in.
Wow! I just checked my PA and DA today and found some "extra points", good to know that they are changes in the algorythm
So I guess you're saying that the "new" PA is better than the "old" PA. Thanks!
PS (You may want to think about doing a summary on top of these blogs for the driver personality types, although we are few in the SEO world, we would love it.)
A summary for this Blog could be:
Summary for Driver personality types and A.D.D. clients:
The New P.A. D.A. factors are going to be evern more accurate starting this month. You may see slight changes based on this.
Great post Matt thank you . Looking for more ..............
Matt, great job!
This blog post is a big contribution understanding PA and DA parameters and it's the answer of my question "Does DA and PA Seomoz parameters have all it takes to replace Y!SE? "
I'd really want to know what you think on what are the parameters that need to be checked before acquiring new links.
thanks ahead!
Aewsome! I love metrics. This kind of analysis is great.
Thanks, I use these all the time too and in conjunction with the keyword difficulty tool they help me to begin to determine what it will take to rank a site.
With that in mind the keyword difficulty tool has always been a bit of a pain because the score it gives out is not relative. How about adding a field to the interface for the D/A and P/A scores and then returning a relative score? It would save me using my rather clinky matrix to figure out ranking potential.
I think other Mozzers would love it too.
Awesome! All I can say is that you guys rock at making our daily lives as SEO's that much easier. Keep up the good work and thanks again! Very excited to see the rollout.
Cheers,Ross
We use PA/DA in the monthly reports, so I will have to write a short summary about the recent changes if there's a significant increase or decrease(!). Clients always check historical DA data and compare...
Great news, I use page authority and domain authority metrics on a daily basis so this update is greatly appreciated.
Happy to see so many fantastic developnments at SEOmoz this week.
Must be one the most colorful posts I've seen on SEOmoz :)
Google’s Page Rank analyser is seriously outdated so in that case PA/DA is really helpful to determine the quality of a website. I use this on daily basis in my research and link building process.
As i am a regular user to this i believe this update would be really helpful.
Great post, Matt.
Awesome post - your diagams make me happy) New model will work much better, i hope.
Fascinating detailled Insights with the principal component analysis! thanks for these geeky graphs :)
Great post, Matt -- packed with some really interesting data. The outcome of the PCA was really fascinating. The post prompted a few questions:
1) Why does the old PA correlation generally increase with each index? Is this an outcome of better (deeper/wider?) crawls?
2) What machine learning is used? Or is this part of the secret recipe? I'm interested to know as it seems the output is direct weighting of the inputs rather than any more complex interactions (if X then Y is important, but if !X then ignore Y).
3) The fact that the new PA algorithm correlates better with 50 entries than with 30 seems to imply that there are metrics that are important to the top end of the SERPs that aren't captured so well than the lower ranking entries, which is interesting. Any thoughts?
4) What's the correlation against mozRank and mozTrust and is it linear? I see a lot of people who treat higher PA/DA as passing more juice, which seems like a reasonable assumption a lot of the time, but sometimes not. Furthermore, if it is normally the case then does it confirm to the same sort of logarithmic scale as mR (I think base on that is ~8.5?).
Thanks!
Glad you enjoyed the post and great questions. I'll try my best to answer them.
1. The Linkscape team has been working hard to improve our crawl and resulting index for a few months and the slightly increasing old PA correlation was a result of this effort. The new PA slightly increases with each index too since it uses the same link information but is always substantially better then the old model. As we scale and increase our index size in the future I expect the correlations to continue increasing.
2. I can't give too many details on the exact model since that's part of the secret sauce, but can say that it's a non-linear model that includes complicated interactions between the various factors. I tried a linear model with a direct weighting of the terms but it doesn't perform as well as the non-linear model.
3. I think it's a general rule that the more results you use the higher the correlation, but don't have older SERP sets with more results to test the older indices. Here's a thought experiment I use to convince myself: let's take two extreme cases, a SERP set with 2 results and one with one million results. We expect that the first and second result will have pretty similar ranking factors but there will be a huge spread between the first and one millionth factors. As a result, we expect that anything that discriminates between results will do a better job with the larger number of results and therefore have higher correlations.
4. mozRank and mozTrust both have correlations of 0.20 on the 50 result set, whereas PA/DA have correlations of 0.36 and 0.25 (our two highest values). Both mozRank and mozTrust are logarithmic. Below about 30-40 PA is approximately linear (Gaussian) and above this it is logarithmic. DA is logarithmic.
Matt,
Thanks a lot for the thorough answers.
I'm not sure I entirely follow answer 3 given that, I believe, Spearmans rho is agnostic to the sample size. I need to digest.
Thanks again for the post and answers -- packed with so much information. :)
Love your metrics, I use and refer to them all the time. Look forward to seeing update changes.
Happy Turkey Day All!
I always use these metrics with the SEOmoz ToolBar. All I have to say is "THANKS".
Also after reading it, I have to say HHHHUUUUHHHHHHH????
Very Nice Post!
I have never used this Tool on SEOmoz to analyze page authority and domain authority.
So I want to have a try.