When Penguin 4.0 launched in September 2016, the story from Gary Illyes of Google was that Penguin now just devalued spammy links, rather than penalizing a site by adjusting the site's ranking downward, AKA a penalty.
Apparently for Penguin there is now "less need" for a disavow, according to a Facebook discussion between Gary Illyes and Barry Schwartz of Search Engine Land back in September. He suggested that webmasters can help Google find spammy sites by disavowing links they know are bad. He also mentioned that manual actions still happen — and so I think we can safely infer that the disavow file is still useful in manual penalty recovery.
But algorithmic penalties DO still exist. A client of mine, who'd in the past built a lot of really spammy links to one of their sites, had me take a look at their backlinks about 10 days ago and build a disavow file. There was no manual penalty indicated in Search Console, but they didn't rank at all for terms they were targeting — and they had a plenty strong backlink profile even after ignoring the spammy links.
I submitted the disavow file on March 2nd, 2017. Here's the picture of what happened to their traffic:
4 days after the disavow file submission, their traffic went from just a couple hundred visits/day from Google search to nearly 3,000.
Penguin might no longer be handing out penalties, but clearly there are still algorithmic penalties handed out by Google. And clearly, the disavow file still works on these algorithmic penalties.
Perhaps we just need to give them another animal name. (Personally, I like the Okapi... goes along with the black-and-white animal theme, and, like Google algorithmic penalties, hardly anyone knows they still exist.)
Image courtesy Chester Zoo on Flickr.
I look forward to animated comments from other SEOs and webmasters who might have been suspecting the same thing!
Extremely short article with only the one case study which doesn't explicit state that the chart shows organic traffic. Really surprised Moz even published this
I agree with this, not surprised OP has replied to all other comments and not this.
Hmmmm. While I respect Michael and his work, I'm not sure that I agree with this article.
There is no way that we can know if this recovery is due to Google processing the disavow, or due to something else. My guess is that it is due to something else.
As mentioned in the first comment by Michael, there was a major algorithm update that happened at the same time. Here are several tweets in which people mentioned massive improvements in traffic with this unknown update (that many are calling Fred):
https://twitter.com/glenngabe/status/8409088545106...
https://twitter.com/glenngabe/status/8409085258911...
https://twitter.com/AlanBleiweiss/status/841391165...
I've also received emails from sites that have had incredibly massive drops in the SERPS over the last couple of weeks.
I have filed disavows for hundreds of sites. While I have seen a good number of recoveries, I don't recall seeing one this dramatic just a few days after filing a disavow, other than one that was combined with the removal of a sitewide manual action.
When a disavow is filed, those links won't be discounted by Google until Google crawls them. For ultra spammy sites, this really should take much longer than six days. The more normal type of disavow related recovery that I'm used to seeing is a much more gradual one.
Still, it's not entirely impossible that this recovery was due to the disavow. It is true that other algorithms use links. But, I think we need more than one case study to decisively report that algorithmic link penalties still exist. Don't get me wrong...I absolutely love doing link audits and, in fact, this article alone has resulted in me getting emails requesting disavow work. But, I think we need to be realistic in our expectations.
I'll report back if I am seeing similar results after filing a disavow but at this point I really am leaning towards saying that this recovery was due to the dramatic algorithmic shifts we are seeing and not related to the disavow.
I agree, Marie...I've never seen a disavow remove a penalty this fast. I'm still on the fence, to be honest, about whether I'm convinced it's a penalty being removed by the disavow file. Of course, it COULD be that part of Fred is implementing a much more real-time consideration of disavow files.
If it was just Fred causing the traffic and rankings boost, I'd expect them to have moved a couple of places up across the board. But I'm seeing 21 spots jump on average. And, it's pretty clear they've been under a penalty for the past couple of years, as their traffic used to be about what it is now, and fell off a cliff.
Michel & Marie, I 100% agree. With only one case study as the example, it's hard to justify that removing a disavow could have such a speedy impact on removing a penalty on the website.
I'd be interested to see if this process works (at all or as well) on another website before jumping to any conclusions.
This is a great thing to consider, and in my opinion disavowing known spammy links can't hurt. The Google "Fred" update around the same time does create some hesitation in acknowledging a direct causation, but the disavow file is still something SEO specialists need to keep in their strategy "toolbelt", especially those with a ton of low quality links.
While it's difficult to pin down the causality of the disavow, I do think it's really important for people to hear that the "kinder, gentler" Penguin did NOT mean the end of link-based algorithmic penalties. Penguin was not the only piece of the algorithm that targeted bad links.
This is first time I have seen such kind of quick recovery (just 4 days) by any site. Although, I have disavowed many website's in last 6 years and I believe it still does matters. At the end you want to rank in Google, so if you will help google to find low quality links that you have built, that would be appreciated by Google.
But yes, we should not relay on Google always that they will make it nofollow every time when we submit a file. So, it's better to avoid spammy links and build only quality contextual links if possible.
Thank you !
Yes...I've never seen a recovery in less than a few months. Although, I've certainly seen a negative impact on PageRank in a few days from a very aggressive disavow!
There's (not surprisingly) some discussion going on about whether this really was the result of the disavow, or whether perhaps 4-5 of my client's competitors all got hammered by the update, and THAT moved them up 4-5 places in the rankings.
I think that's possible...but, in Search Analytics, I'm seeing an average ranking for a big group of their terms go from 21 up to 7. And, their traffic fell off a cliff a couple of years ago, making me think penalty even more.
Great Post! Very interesting! I have noticed this as well. What is classed as a spam link though? is it because the content is badly wrote? is it because theres too many links linking to the backlink. Or is it because there is very bad technical SEO like page speed?
How can Google just ignore links? Personally I think that this is a distraction from Google. They are a very clever company and I think that this Penguin update won't last long until it penalises you.
What I looked at as spam was mostly obvious link farms, easy-to-spot SEO web directories, and links from sites with malware.
Thank you Micheal
It would be very rare if this is a result of a disavow. Google has to crawl those disavowed links first, and if they were low quality then the would not be crawling in 4 days. Unless, of course, you forces them to crawl them.
If they were paid links on really good sites, then it is possible they were being crawled every day... thus that disavow would help.
Hey Bill, good seeing you at Engage in Portland! I'm not convinced Google has to crawl the disavowed links themselves, though. The process might very well go like this:
It seems to me that Google would have to crawl the links if they were removed, or nofollowed, but not disavowed.
But of course I don't have any actual visibility into the Google code :-( so I'm just speculating.
Once Google has crawled the submitted disavow it shouldn't take long for results to change. I had an issue with a submitted disavow which included a REALLY high quality link someone submitted. 2 weeks later I removed the disavow and it took 2 days for it to return. Are you sure google has to crawl the links? It makes no sense if it already have the info in its index.
I completely agree with Drlovecherry here. A disavow essentially tells Google not to BOTHER crawling those links since you don't want them counted anyways. Google only has to recrawl if you've managed to get links REMOVED from a site--then, they need to recrawl that site and see that the link is no longer there. Not the case with a disavow!
From what I can see the disavow controls what to consider in its index, I believe crawls are independent and their frequency of crawling wouldn't be effecting. Plus, when you submit the disavow you are directly submitting it, you don't have to wait like a usual crawl, the changes usually take place in a day or two.
Michael, I believe you are onto something but like other people said, you can never be sure. Then again, isn't that the case with all SEO? All we have is indications. Ignore PIXUS at the top of the comments, his comment seems quite rude. I appreciated the read, although it was a little short. It is always nice to see the differences in results people have around suspected updates.
I'm going to disagree with you here Michael. In Google's documentation on the disavow tool they say the following (emphasis mine):
It may take some time for Google to process the information you’ve uploaded. In particular, this information will be incorporated into our index as we recrawl the web and reprocess the pages that we see, which can take a number of weeks.
Marie, I enjoy a good discussion. Maybe this information is out of date or a clause. Either that or I was extremely lucky in the 2 days it took to resolve the disavow issue I faced. I just don't see the logic of it being a big part of indexing. Our disavow files are almost like a filter, which is applied to a data set before assigning rankings. Why would Google have to crawl the pages you disavow before affecting rankings?
It certainly is possible that things changed with Penguin 4.0, but Google employees have said many times that the way the disavow works is that as Google crawls the web they'll process the disavow.
It sounds like you are saying that you realized you had a high quality site in your disavow, so you removed that from the disavow file and then rankings improved 2 days later? If this is the case then there is a good possibility that this is a coincidence. Matt Cutts said a while back that there is a lag feature built into the disavow so that if you re-avow a link it will take a while for it to be counted again.
It seems unlikely to me that disavowing or not disavowing one link would make such a big difference, but I could be wrong.
In this case it was a high value domain that linked to us a few times from hi PA pages to key pages on our site and it had a drastic effect. It was the domain that was disavowed, not an individual link. Maybe it is a coincidence, I will never know. Maybe the way they handle disavows has changed and not been updated?
When it comes to Matt Cutts I don't trust everything he says. Obviously he does point people in the right direction but I only making decisions based on data rather than "the word". I appreciate your feedback on the subject, I don't want to come across as rude in discussions when it comes to conflicting opinions.
This is great Michael, thanks!
Just wondering what sort of sites you disavowed? Were they general spammy sites or paid links. Do you have a few example we could see or some more specifics?
Also, what sort of website is your clients? Obviously you're not going to share real site just interested what sort of website it is.
Also, I'd love for you to give us an update on whether the traffic changes again in a couple of weeks.
Sorry I've got so many questions, just really interested in this case study :)
The client's site is a directory of government offices. The types of links you'll see in my response to John-Paul, above.
Sample sites:
Hi Michael,
Very interesting, thanks for sharing!
I think the fact that the disavow tool still exists shows that it can be useful. I have had a few clients come to us with some very spammy backlink profiles and we do our best to help but when it is very bad we use the disavow tool. I do not usually see improvements like you saw, but it gives me peace of mind knowing we cleaned up their back-link profile.
Hi John-Paul,
I agree with your approach. I'd expect that there are plenty of sites out there that are very close to the threshold of receiving a penalty (manual or algorithmic). It's a lot easier on your heart, your ulcer, and your client to be "ahead of the game" and keep the backlink profile relatively clean.
Having said that, there's certainly a risk of disavowing too much. There's a lot of links I'd consider spammy that are getting counted and helping sites' PageRank, and disavowing those will hurt the client. I tend to disavow just those that are (a) super weak...so they're not passing much link juice anyways, and (b) egregiously, obviously spammy in such a way that Google SURELY is categorizing them as junk. I.e. groups of big SEO web directories that have hundreds of thousands of listings AND you find they all look almost identical except for the logo; articles that have been republished on dozens of sites and originated in ezarticles.com or articlesnatch.com or one of those article marketing sites; or sites that Chrome spots as containing malware.
Thank you Michael for the informative and awesome content. Yes, Google algorithemic penalty still happens, though algorithms are made to check specific purpose and they does it perfectly. They will be more effective and perfect with time.
I personally think Google takes some time in verifying those disavow links submitted by you. But 4 days recovery, is really a quick one.
The increase in traffic in that example is quite impressive! What % of link did you disavowed on this website?
Question. Should a link coming from a dead website(hasn't been touched in 10-15 years) be considered to be disavowed?
203 domains disavowed out of 1106.
Regarding old crusty-musty links: the rule I go by is: if Google Search Console still shows the link, they're probably still considering it in both penalty calcs and PageRank.
This is interesting. I am not sure I agree with the assessment. Several Google employees and blogs have stated it can take weeks if not months to review a disavow file. While I can be processed in 48 hours, it is highly unlikely. My client's website saw Yuge spikes in Keywords ranking increases and we have never submitted a disavow file. The quickest I have seen a website recover from a penalty was a week and that was RapGenius. I have seen a lot of people point out the update on 3/7-3/8, which correlates to the increase in sessions. I have spoken with several site owners that have confirmed Yuge increases and decreases in KW rankings these dates. Barry "Rustybrick" Schwartz has claimed it be geared towards site content. I agree with his assessment too. I have seen websites impacted by Panda updates see a drop in rankings and traffic.
Having seen an interminable amount of this site's content, I'll have to say I really doubt that any Google algo change worth its salt would give it a boost for its content :-p. It's just pages of a little text content and then links to government reference sites. UNLESS....Fred is all about giving giant boosts for outbound links to very trusted domains!
On the timeline...some thoughts here:
Many of Google's big algo processes have recently been integrated into the algo near-real-time. So possibly algorithmic link penalties got integrated as part of the Fred update.
Also, I've definitely seen a case where a site fell in rankings just a couple of days after submitting a really aggressive disavow file. So, clearly at least SOME processes read the disavow file within a couple of days.
I wish it was the output of a penalty. That would tell us that Google is faster to remove them. Thanks for the info.
Links aren't just used in the penguin algorithm, Google still advises you to use the disavow file:https://twitter.com/JohnMu/status/7793047429294653...
I'm going to share a screenshot of my experience over the past few weeks as well.
We've had a running disavow file since January 15th, adding to it almost daily to fend off a negative SEO attack on my site. And, we were still seeing search traffic and rankings decrease significantly.
However, notice on March 9th the uptick in search analytics. We submitted a new disavow file that day as well, and it's still trending upwards. We've seen our positions drop from 1-4 on Google down to ~16, and now we're back up to average position of 6.7. Good to see some recovery, hopefully the trend continues.
screenshot: imgur.com/a/6AFGj
Actually I discovered I have another client as well who's seen a huge uptick right around March 8th, after fixing a problem with sitewide links from some partner sites. I think they moved up 40 places. And it's not a site where content would have made a difference.
Where can we see our backlinks and filter them?
Thanks.
Go into Google Search Console. Under Search Traffic, click Links to your site.
Under "Who links the most", below the list of sample links, click More.
Then click "Download more sample links".
Interesting article and case study, though I am not convinced either. The time period between the disavow and the recovery leads me to believe something else is at play besides the disavow. I have yet to see recovery time this quickly. Thanks for publishing your study though.
I think it's a good article, but rather brief. We must always remember Penguin not to fall into the temptation of black SEO
thanks for sharing this! Really interesting. I remember before they made penguin real time Cyrus Shepard showed how he had disavowed all his website links and didn't recover until the next penguin update. So could it be that links indexed before penguin 4 are still susceptible to link penalties? Maybe only links indexed post penguin 4 are devalued? ...how old where the disavowed links?
Hi Matt, that's an interesting thought. These were definitely very old links....4+ years.
Thank you for your article. I think my web is penalized in Google :(
Look in Search Console's Search Analytics for a sudden drop in rankings on a given page or set of pages. That's generally your first clue.
We have over two million follow links spread over thousands of domains pointing to our site via a "powered by" footer. These sites are using our web software and they have been around for many years. A couple of years ago we changed the powered by link to nofollow, but the older sites that didn't upgrade are still out there. We also changed domains right about the time when we added the nofollow, but we had to redirect our old domain to the new one so the old links (pointing to our homepage) are still valid. How can we know if these links are negatively affecting our traffic in order to disavow them?
If there's a penalty, it'll be dramatic...typically I've seen a loss of right about 40 places. It's been consistently about that number, which makes me think that penalties aren't implemented as a loss of PageRank in the calculation, but rather a push-down of a specific number of places (or completely gone from the results). But that's just a gut feel.
Is this example based on a backlink analysis via Search Console, exclusively?
Yes. I'm only ever looking at backlinks downloaded from Search Console. In my experience, Google finds more links than any of the other backlinks tools. And, I've never seen a manual penalty reconsideration request rejected based on a link that WASN'T in their list in GSC.
Hi Michael,
while I support the claim that working only with GSC data is the best start for free, I'd like to add some thoughts due to my experience with penalty analysis based only on GSC data:
1. Make sure you have all entry hosts of your domain set up as a property in GSC, e.g.
Then download and aggregate all link data to make sure you don't miss out on some links.
2. Take in mind that deleted or deindexed links might not show up in the GSC link export, while being good indicators for the history of a penalized site. I've also had a case in which Google was showing an example link that wasn't part of the link export.
Just my 2 cents.
Best
Chris
That's a very important point, Christian. Interesting that you've seen an instance where an example link wasn't in their list!
Michael, I generally pull link profiles from Moz, GSC and Majestic (and usually add ahrefs if it's a large site) and combine the results. In every case, each of those will unearth a few links that none of the others do. And I've never had a recon request rejected that way.
How to find disavow or bad back links to my site freeeeee ? is there any tools
I use Kerboo's LinkRisk to evaluate backlinks. It's about $3000/year though. But I find it's far more accurate than, say, Link Detox, which I found gave a lot of false positives, and let slip through a ton of really obvious spam.
Pure coincidence. As usual, we don't have access to Google's black box so this 100% speculation. Didn't you notice the buzz at blackhatworld and seroundtable?
https://www.seroundtable.com/google-fred-update-ad...
Of course....read my very first comment :-)
Certainly it's speculation. But the evidence has me mostly convinced it's the disavow, because of how many places they moved on average (21), and honestly their content is pretty thin, so it's unlikely to be a boost because of their content.
It also could be an ancient penalty that simply expired, and that happened to coincide with the Fred update. We are pretty sure they were penalized algorithmically (no manual notice in GSC) about 4 years ago.