Anyone who's been in internet marketing long knows the feeling: you go to check your rankings one day and something's different. Your pulse quickens, a cold sweat spreads across your brow, your eyes widen...they're gone. Your precious rankings are gone! Where did they go? What happened?
Penalties. They can bring the strongest of us to tears. But sometimes we can be too quick to blame a penalty on sudden changes in our rankings. There are lots of things that can cause fluctuations (even major ones) in the SERPs and it's importnat to know how to figure out why you've dropped. This week, we'll look at how to identify whether or not you've been penalized, as well as what to do if you have been smacked down by our Google-y overlords.
Check out this week's Whiteboard Friday and next time you're struck with a moment of SERP Shock, you'll know how to handle it in a cool and strategic manner.
If you want more information on penalties, here's a slide deck Rand presented at a recent conference....
I like how the title of this post uses "$#!%" to represent "shit." Most folks would simply type a random symbol for each letter in the curse word, but you went the extra mile by selecting symbols that look like the original letters ($ = S, # = H, ! = I, % = T). A lesser site would have neglected these subtle details, leaving its readers to wonder things like: Does that say "Oh shit, I got a penalty" or "Oh fuck, I got a penalty"? Is it possible that those symbols don't even represent a curse word at all? Could it say something like "Oh good, I got a penalty" or perhaps "Oh Matt, I got a penalty"? Thanks to SEOmoz's attention to detail, I don't have to waste any time pondering such things. As a side note...I'd like to recommend the plus sign as a better alternative to the letter "t." For example: Oh $#!+, I Got a Penalty
you waaaay over thought that response ;-)
thumbs up from me all the same!
Darren, your irreverence so earns you a thumbs up. You are truly a crazy SEO mofo.
Haha, was that the biggest waste of your time today? :D
It's Friday, £0#! it.
classic
Thanks for noticing :)Damn, wish I'd thought of the +
that's really really funny...but I don't know what's scarier- that Scott actually DID think about it, or that you noticed it.
Really useful Rand. I got a few take away's from your WBF as well as the comments:
1) Let my hair dry before donning a helmet
2) Jane is a spammy link expert
3) You taught Jane everything she knows
4) It's safe to make Catholic jokes in Italy
5) Bookmark this post in my RSS reader under "Google Penalties"
This will be a valuable resource when I get a client who's been smacked aside the head with a Google 4x4.
* Because she is very good at avoiding them and has had to clean up other people's messes a couple of times before :) Just to clarify.
Ohhhh. I thought it was because...I mean I thought he meant...umm, I gotta go. I think I hear Sly-grrs mother calling me.
Speaking of getting smacked aren't we due/overdue for another "Florida" update pretty soon?
I thought Rand was the expert and taught Jane everything she knows about spammy links
I know you kind of jumped past the first part for this topic..."I'll assume you did X, Y & Z," but I just wanted to toss out three things that I think are major first steps:
I can't stress the last bit enough.
If you bring on an agency to help you specifically because you think you've been dampened or penalized, consider your SEO agency as part clergyman, part attorney...you must come clean and confess your sins, minor or great, if you have engaged in anything that might be questionable.
Couldn't agree more with your 3rd point. Also because without this first stepwhat kind of trust can be build?
Next time I will meet a client like this I will wear my catholic priest carnaval costume and tell him, after his confession:
I forgive you in the name of Google. Pray two robots text and a Sitemap.xml.
PD: just to avoid any complaints, the joke is not intended to offend any catholic SEOs here (it's just italian humour... as we have the Pope at home).
Nice.
I realize I'm going to hell now.
ROTFL. Nah identity. You were going to h--l way before this :)
;) yeah, I figured it wasn't a huge risk! That's ships most likely already sailed.
... and you can imagine my surprise at being spoken to directly from my montior first thing in the morning!
Good video, Rand. You taught me everything I know, right? :D
I used to take that as a compliment, now I'm wondering... :-)
BTW - Love how other comments on this post aren't asking me for help, but rather just go directly to you.
Interesting question about wether or not to abandon a domain and start over.
My experience (from several penalty recoveries) is that if Google can see some real value in your site, you'll get back after cleaning up + building some new high quality links.
But if your site is some-what-spammy-not-value-adding one, I think chances of recovery is slim.
Another potentially helpful way of dealing with a terminally penalized site is 301-redirected to a new domain. At the very least it won't hurt (otherwise our competition would be setting up fly-by-night sites, black-hatting them into getting banned from the index and then redirecting them to us).
It's exactly right, as Rand hints, that a lot of perceived Google penalties are not penalties of the site itself. This is especially true when the drop in rankings is caused by offiste factors such as backlinks. Google simply devalues spammy links. (But if backlink abuse adds up to a prominent pattern bad and large enough to trigger manual review by Google, then a serious penalty may indeed ensue.)
All that said, I sometimes am not sure myself to what extent arguments from competition are convincing. In this video Matt Cutts seems to say backlinks cannot hurt you (though admittedly one needs to be careful about drawing this conclusion with confidence). On the other hand, I am reminded of this old interview of Cutts that Rand conducted where at 9:35 Rand says that he thinks Google can penalize sites for engaging in linking schemes and Matt emphatically nods and they discuss footprints (without using the word) and IP addresses and such. And that was some years ago. I wouldn't be surprised if it turned out now that Google has become much better at distinguishing us confidently from our competition. Surely Google has the technical ability of keeping lists of IP addresses associated with any site if it wanted to (think associations via Webmaster Tools, Analytics, AdWords, Gmail...), and track activities from those IPs via the toolbar, Chrome and other gogooglies. Tracking such stuff would be expensive, but it would ultimately reduce SERP manilpulations and "improve search quality" -- and Google has a lot of computational brute force at its command. Am I being paranoid in suggestion this? Is Big Brother not watching? I am not a blackhat and not worried, but would still like to know.
Rand, Jane & SEOmoz, do you have a candid view on this? If so, please share.
Yea definately another great WBF!
I also would like to comment on Rand's great use of imagery in the slide deck presentations, always a laugh! :)
Hi...
I really like the last suggestion you gave (restart from zero).
I like it because, being an externarl consultant, it happens quite a lot to me to receive inquires by "little guys" who are contacting me with the classical question: I cannot find my website on Google if not using the domain name of my site (or worst...)
I took that petition of help as an indicator that something wrong is going with that website. And with that suspicion in mind is then quite easy to check out if some black/darkgrey hat tecnique is used on that website.
But, your suggestion is great especially because what I don't know is the penalty history of that website... I don't know if it was banned more than once and so on. And many times (strange but true) I cannot have any Analytical data to examine to investigate the past of the website.
Therefore my strategy in these cases - as you say - is to suggest a complete reinvention of the website: the time the client will loose waiting for a not so sure re-inclusion in Google can be better used in order to plan a good white hat SEO and Web Marketing Strategy.
On a final note:
You can ask the client about the seo work done in the past. Usually client has some ranking or audit reports from the seo he last hired. You can then compare these reports with server logs, to find out how the seo work impacted the site. Archive.org can also help in finding out what changes were made to the site.
I can assure that not all of them.. especially if they were SEO no savy (and still they are) and victims of pretended SEO agencies.
Another classical issue is that maybe they had GAnalytics... but the account was held by the previous SEO not by them... therefore no access.
About ranking reports: they can give hints (for instance if you notice a sudden ranking in 1st page from page 100).
But I agree with you that the server logs datas, Archive.org (again, not always) can be used for an investigation work.
Hi seo-himanshu. Can i ask you something if it's possible? You've mentioned that i can compare my past seo report with server log to find out seo work impact. After reading your comment, i accessed my server and tried to find Server Log. The closest thing i found is Access Log that shows me who and what has accessed my site.
Can you explain little bit about the term ''server log'' you wrote?
Thanks a lot :)
Really nice video on the differences between lost link value, traffic drops, penalties, etc. We recently posted something similar in our blog as this question has come up a fair amount of times from perspective clients & current clients. Bookmarked. Thanks!
Hey Wil! Long time no hear from. I'm going to go out on a limb and ask the question that I know everyone is wondering...
Are you trying to bring ear muffs back into fashion?
'Cuz that's a slippery slope if you are. Next would be those neckties that were wide enough to do double duty as barbecue aprons. I'm just sayin'.
I've had this happen to me several times for multiple sites - removal from the serps all together or pushed to like page 8 for up to 2-4 weeks at a time. My advice is, if everything checks out and you know you didn't do anything bad, to just ingore it as hard as it may be. EVERY time in my case things went back to normal.
I haven't run into this lately. (We've been getting good clients this year) but there is something my team calls "Automatic penalties"
It looks like Google can automatically detect certain spammy behaviors like loading your footer with a ton of SEO-optimized anchor text links.
When we see this behavior and either take down or reduce the number of links, the penalty is lifted within a month or so. No reconsideration request is needed.
I don't see this mentioned anywhere so I thought I'd throw this out there to see if others have experienced the same thing.
Say a relatively 'small guy' website does naughty things and the domain is burned. The naughty company decides to create a new identity. The domain expires or maybe you as a competitor notice they are no longer using the domain. You see value in buying their keyword rich domain but you're worried the domain is still scarred from the pervious issuer. Is this a legitimate concern that Google will not be able recognize you’re a different company that does the same thing? Does the stigma follow that domain into purgatory?
I don't have a definitive answer, but I would hope they'd notice the new owner, the new IP address and the new copy.
Agreed - most of the time, you'll be OK, but it never hurts to take a domain you've just bought and register it with Google WM Tools, then send them a re-consideration request indicating the site is basically "new" to you, and you want to make sure they know none of the previous links/tactics used were yours. This pre-emptive step may help you out of future jams.
You can use the IP: command on bing to find out whether or not the IP address is blocked by search engines. whois.domaintools.com can be used to check out for bad neighbourhood.
Thanks for the slide deck Rand. I think one good way of evading penalty is to keep an eye on the websites you link-out to. Who knows when a legitimate website of today becomes a spammy website tomorrow. Therefore it makes sense to request the client not to link out to any website without consulting his seo first. Keeping record of all the changes made to a website esp. those changes which can have dramatic impact on ranking (like changes in the title tag, robots.txt, .htaccess etc) on daily basis is the best way to deal with penalties. So if you see a sudden drop in rankings or traffic tomorrow, you can immediately check out what changes made to the website could have been responsible. Here is another article from seomoz, which goes very well with this post:
https://www.seomoz.org/blog/how-to-find-the-spam-youre-linking-to
Keeping a record is an excellent idea. Just yesterday I went to change a clients theme on his website and I completely forgot what I did the first time. And I had no record. bad me!
So before I change the theme today, my first action will be to record what I do.
It snuck in quietly, but the notes function in google analytics is very handy for this (a little restrictive size wise, but great functionality none the less.)
Seeing your notes simply and easily with traffic variations is very powerful and it also leaves an open record for anyone else working on the site
from a client side, id put it in my contract that seo changes need to go in here. keeps seos honest ;)
Awesome whiteboard friday, I think a lot of people assume that the value of links will always remain the same when they do not and this is particularly the case for some really competitive terms as opposed to the long tail (I have noticed at work). It's something that worries clients a lot and it's hard to convince certain clients that it's just something that happens and you have to just keep going and 'get back in', like you explained.
Edit: By the way, you got cut off when you were saying 'whiteboard friday' at the end of the video haha.
Trax buddy! Nice shiny new avitar!
Yeah I thought I'd go for a more personal photograph! Everyone else seems to have ones of themselves, so why not me?
That excludes you of course. Unless you're canine.
He wouldn't be the same without his doggy avatar... :)
I can't bring myself to change his photo. He was the sweetest yellow lab I have ever had. His name was Boozy (pronouncing the o's like in "wood")
Besides, if I used my photo I'd probably break a few monitors.
Lovely tribute indeed to your best friend making it your avatar in SEOmoz. Seriously, from the proud mascot of a cat (it's not a mispelling, simply the truth).
Awh that's really sweet :-). I've never been lucky enough to have a little best friend like that. I need a dog.
and i just thought you were trying to keep your identity a secret, i could suggest glasses hell it worked for superman...
Over the past month or so I've been helping nurse a website better. Made reference to Jane's slides from the London seminars last October (which she kindly sent me again).
I created link profiles for ~7 competiting sites including my clients and compared the anchor text distributions. Discovered that previously whoever link built before, did so with the vast majority of anchor texts featuring just the vanity term, leaving a huge disparity between the vanity term and all the other anchor texts. Of course this looks completely unnatural and is probably not doing us any favours. Now working to amendment the link profile.
In conjunction, I've been clearing up some 'toxic links', and would seriously appreciate some feedback from people (especially Jane) on how toxic links affect link equity.I've identified 'toxic links' by looking for huge disparity between mozrank and PR, combined with a poor link neighbourhood. So far when removing them though, initially appears that they do pass a very small amount of link juice, but not negative. My questions
My findings suggest me to think that removing such links does not have an impact at a micro (link by link) level, but rather at a macro link reputation level. It's proving really hard to measure. If anyone has made any discoveries to suggest toxic links can pass negative link juice, or that it contributes to a 'link reputation', I'd be interested.
I think the latter would make much more sense for Google. For links to pass negative link juice would mean that individuals could quite easily 'toxic bomb' a competitor. However if Google profiles a site's link portfolio and makes a macro judgement, that takes away such power while Identifying remnants of poor SEO practice.
Answers/thoughts on a postcard please :)
So what's with this new theme: Jane Copland and toxic links?
Thanks for the introduction to Jane, I spend a lot of time analysing links and cleaning up old SEO black/ grey hat techniques left on sites I have taken on to manage.
Its a scarry thought to drop off the radar for our own web design site but explaining this to customers without sensible reasons would be difficult.
Hi All,I have a question that may lead to some interesting discussion.Would a website that has 'dropped back' a 2-3 Pages for a particular keyphrase be a sign of a Google Penalty?
Is a Google Penalty defined as an entire domain being banned from the SERPs? or an Individual Ranking?Or in the case of an individual ranking, it is the case that a proportion of links (with that anchor text) have essentially lost value.
Thanks for sharing this deep insides.
That sounds oh so bad.
Hey Rand, great WBF...you mentioned in the beginning about a blog post re: profiling webmasters or SEOs. Were you talking about the one from June 2009?
Unfortunatley I'm stuck in this very situation at the moment. My site was in the top 3 on google.com at the start of this year but since then I've been trapped in a cycle of being relegated out of the top 1000 results, only to be promoted again after reconsideration requests.
The most infuriating aspect of all this is that I have absolutely no idea what may have caused the problem. For me the past few months have been quite stressful, however, I can't even begin to imagine the damage which this could cause a small business which has come accustomed to steady traffic from Google.
The worst thing about it is that you're never made aware of the charge, let alone the evidence. Surely there has to be a better /fairer approach to this problem.
Oh, in case you're interested, my site is:
www.ishigaki-japan.com
It used to rank at between 1 & 3 on google.com for the term 'ishigaki japan'. It's the most comprehensive guide to the Japanese island of Ishigaki which I'm lucky enough to live on. (In fact, in all fairness I guess you could say that it's the only guide).
You may have seen lengthy discussions about my problems on google's webmasters forum.
How come you don't post these on youtube anymore, I got no sound on my computer and auto captioning would be great?
oh wait, I just realised the transcribed text makes almost no sense
Excellent and insightful video, thank you! This is something I luckily have only had some experience with, as I was trained by individuals who actually know what they are doing (whitehat).
One question - In your experience do you find that Google is more likely to penalize a full domain, or a specific sub-set of keywords or pages?
I ask, because we have seen large rank shifts around keyword groups or pages all the time all the time.We had one client drop from 9-10 range all the way down to 90-100 in a day last week, but the rest of the site looks good.
However, I usually don't attribute these instances back to penalities, moreso lost links / changes to algo's / etc. Also, I know this might depend on the circumstances, but I wanted to know if there was a general rule of thumb that first they penalize a page, then the whole site.
Thanks again!
Just a topic I needed to read about (or watch some video :))
This happends to me just a week ago, my rankings droped, my traffic droped (95% of traffic come from google) and I dont know what to do...
That's also a warning about not being overly dependant by Google and that a good strategy is to wide your traffic sources.
Not sure if you had to start again you would 301 your penalized site to your new site.
Would the 301 launder your dirty links?
Whether to 301 or break the ties entirely and discard the burned domain is something that should really be determined, again based on the specifics, otherwise you may taint the new site as well, which would obviously defeat the purpose.
How would you know? If the site is penalized for bad links, I guess you wouldn't want to 301. But say it is penalized for cloaking but has good links, that would be a case where you would want to 301?
That's just it, how to handle should really be determined based on specifics of what has been done and data. Ideally and hopefully 301's could be used.
Rand, you mention to just "start over" and 301 the old domain to a new one. Do you think that this will just pass the negative factors on to whatever domain you 301 to?
Erm...that's what I just said...
Good assumption about copyright.
Sorry about that, should have read all the comments more closely.
Usually no, but sometimes, particularly if you control a number of these "manipulative" sites, it can happen. My experience with a site that's been burned is that 301'ing may not pass the link juice, but it also won't pass the bad reputation/spam association (at least, a majority of the time).
I agree with Rand. If 301 start passing bad reputation then anyone can 301 hundreds of spam sites to his competitor's site.
Awsome WBF Rand. This clears most of the things i wondered about penalties if not all. I'm just hoping that i won't ever need to use what's in this video ;)
So I recently got a lowered ranking in Google for a set of very important keywords from one of my sites. I don't think it's a penalty per se, since I still have many other pages at #1 in Google. In your video you mention that maybe some of the links coming into the site are just not valued like they used to be. That seems likely, however I have no idea. So finally I'm at my question:
Is there an easy way to track all the sites that link to you over time, so you can see what's going on? Clearly it's not something that I (or anyone) can do by hand, or just remember.
www.opensiteexplorer.org is pretty good for this, and so is Google Webmaster Tools (for showing links to your own site). You might want to try Dr. Pete's technique of analyzing/charting the quality of links that are coming in, and take a hard look back at your link building campaigns, too - oftentimes, you can just identify it based on analyzing the links you've built/bought/acquired.
The Dr. Pete's tecnique is two post back from this one if you missed it.
Gianluca, if there were an SEOmoz award for the most helpful commenter, I would vote for you.
If you weren't a Boy Scout as a boy, I'll be a monkey's Uncle.
Not a boyscout... which is the name of the "Boy Scout" association where the nephews of Donald Duck were in? I was in it when a child!
That would be the Junior Woodchucks ;)
Exactly! I think my mother still have lost somewhere my coonskin cap.
Dr. Pete's technique is just what I was looking for. Thanks!
What should I do if I really have no idea why I was penalized? I've never bought links, cloaked, or done anything that violates the Google Webmaster Policies.
The nature of my penalty is also unusual. Googlebot went from crawling my site tens of thousands of times per day, to less than 10 crawls per day overnight. The crawl rate has never recovered, and all my pages that were indexed in Google have been slowly dropping off Google's day after day.
Anyone have experience with that?
Throw us your website URL, and I'll bet a ton of us get sucked into having a look to see if we can figure it out before anyone else does.... :-)
I got this post today when searching this on Google. Video is too older, but still very effective. Thanks Scott for discussing it.
Im pretty new at all this. My SEO girl tells me i need to build links. So how can a "brand new to this" guy make sure he is building good links and not bad....
I realize most know the answer, but I have no idea
Hiya SynergyLighting. There's an SEOmoz Linkbuilding guide that's right up your alley, but it's only free for Pro members. It's $39 for non-pro members.
If you decide to spring for it, you might consider doubling the amount, and getting a pro membership for a month. That would give you access to all the other guides as well, as well as the ability to ask two complex questions from the Q&A staff.
Then, if you decided it wasn't your cup o' tea, you could just cancel the membership.
Some basic tactics:List your site with S Engine directoriesBlog about it or have other bloggers mention itTweet about it (include the url)Ask your 'reputable' clients (if any) to link to your site. The higher the site's rank that links to your site the better.Publish News Letters if you can ( don't forget to mention your url and include at least 3 keywords relevant to your site) Avoid "link building sites" at all cost. Google will 'sniff' these types of links because most of the time, the links are placed on sites with content that is not relevant to your site's content subject
Domain has been penalized by Google for Hacked content. I remove the Penalty and submit reconsideration request to google webmaster and its approved, But still its not indexed by google from last 2.5 months, even i have already submitted sitemap.xml and robots.txt file.
I have already pushed Google to crawl my site through the Search Console, like Fetch as Google and sitemap.xml submitted to Search Console.
Please Help @Moz
Very good video. Many sites are getting penalty from google. And as you said it may be due to drop in link value.
Just don't do blackhat stuff and you'll never be penalized.
Not correct, sorry. :)