Search engines reps have been calling out web spam individually on their sites for some time. Tim Converse used to do it. Matt Cutts has done it plenty of times, Brian White's done it now, too. I believe there are several inherent goals here:
- Show the web-savvy and blog savvy thought leaders in the sphere (who pay close attention to every search engine rep's blog) that you're smart enought to catch it.
- Create fear of spamming by illustrating the reprisals you'll take against those who do it.
- Talk about it because, hey, that's what you do for a living and you want to share.
I think most webmasters place a lot more weight on the first two motivations than the third, but if you were reading a carpenter's blog, you might fully expect that he'd show you a crappy hammer vs. a great one and probably tell you which brands they were. Do we judge SE reps unfairly? Maybe a bit, but that's OK - they get paid (a pretty solid amount) to take the guff, and I think when you talk to them in person, you'll find that they might even take a bit of pleasure in the drama that's created.
Ever since I've joined the SEO industry, there's been a long-held code that you don't out spam. That code existed as solidly for pure white hats as it did for inky dark black hats, and it's an interesting one. I think the reasoning is that we should all stick together because it's an us vs. them kinda world in SEO. Whether that philosophy (and the underlying code) still holds up today is doubtful. I think there are a lot of folks who report spam, both those who are deep inside the webmaster industry and plenty who simply found spam while browsing the search results and want it gone.
At SEOmoz, we don't report 99% of the spam we see, for a variety of reasons:
- We could spend all day doing it. Once you know the SERPs to look in or the advanced searches to run ("texas hold 'em site:edu" anyone?), it's way too obvious.
- You'll have a far greater return on your productivity time optimizing your site, building content, getting links and conducting press & marketing than you will reporting your competitors for what looks like a paid link.
- It's been done before. From talking with reps and reading in the search world online, you get the distinct impression that the search engines know about spam - at least the great majority of obvious spam. They aren't trying to manually throw out sites one by one; they're trying to develop algorithms to find patterns that will elminate spam on a large scale.
- That 1% of spam that we do report is because we blog about it. When something's really interesting in the manipulation space, we want to share it with everyone - not just because the search engine reps will catch it, but because it often gives insight into techniques we can use in-house and strategies our readers can employ, legitimately, to rank better.
As fascinated as I am by the spam world, I've never wanted to make the quick bucks with it. Some of this can be attributed to fear of the unknown or to my lack of expertise in the arena, but philosophically, I'm not into it. This isn't because I see spam as evil or morally wrong - it's like taking advantage of loopholes in the tax code to keep as much of your money as possible and has about that level of "evil" associated with it from my morality POV. What really keeps me from spam is that it's a short-term solution to the issue of earning money, reputation, building a business, etc.
For those who remember my spam debate with Earl Grey (oh, how I wish he still commented here!), it was all about the long-term vs. short-term. I'm young - 27 years old - and I have the next 3-5 years to build a business that will be sustainable and valuable for the rest of my life. If I were in my 40's and looking to cover my kids' college tuition and pay off my house and buy a place for my mother-in-law, my outlook would be very different. As someone who has 10-12 hours a day to give to my job and loves what he does passionately, I'm seeking something that will build a brand, build a company, build something bigger than a revenue-generation system, but instead something that reaches and helps millions of people.
I'm not saying SEOmoz is entirely altruistic. One of my big goals with the company is certainly to earn money. But, I'd rather have a 10% chance of making $50 million dollars 5-10 years from now with a great idea and great execution and a long-term model than a 90% chance of making $25,000 a month from spamming now, with diminishing returns in the future.
OK - Got a little off-topic there, so back to my original points.
The search engine reps, in my opinion, would do well to call out even more of the spam they find. I think that showing off all the devious networks and telling webmasters that particular paid links or link brokers or spam rings or hijacked pages have been shut down is a good thing. Not only do you get to accomplish your goals of creating fear of spamming (since you're so good at finding it, webmasters would be wise to spend their efforts elsewhere), it also gives legitimate marketers examples of the intelligence and operations of the engines - which isn't a bad thing. I'm of the mind that Matt Cutts could come right out and reveal a rough take on the Google algo and really, he'd be helping Google's index quality, helping legitimate webmasters and sites to optimize and probably not helping spammers at all (or, at least no more than he's helping all the other sites on the web). Of course, he might be helping Yahoo! & MSN & Ask a bit, too :)
What do you think?
"The search engine reps, in my opinion, would do well to call out even more of the spam they find."
Interesting point of view, Rand; I honestly go back and forth. I take little/no joy from calling out specific sites, but I do think that there can be some illustrative value for people to see just how far out there some SEOs will go. I've been swamped lately (I'm taking a few weeks off in May, and I've been trying to get everything in good shape for that), but I've been thinking of circling back around on my "how to report paid links" post to give a few examples of the sort of things that Google would like to hear about and that I don't think add much value for anyone.
No promises, but I'll try to get to it if I get a chance. But I haven't even written about Colbert yet, so you know I've got several need-to-talk-about things on my backlog. :)
hee hee Colbert :)
Matt - I heard he's the greatest living American (won't bother with the link since he's already at #1 and it's condomed) ;)
Seriously though, my feeling on the issue is it might be worth us reporting some things, but I generally feel (as Rand said) that Google knows about most of them and is simply working on ways of improving the algorithm rather than wanting to manually futz around with things. What would be very useful to me is authoritative comments from Google (read, you) about new forms of spam or "great ideas"(tm) that people have come up with that are likely to get them banned or hurt their rankings. It gives me ammunition when speaking to clients / potential clients when they suggest some wacky scheme or tell me about something they are currently doing to say "no, that's a really bad idea". Helps us stay whiter than white, which I'm sure is what you want ;)
Matt - I understand the human perspective of not wanting to embarass someone, I suppose. You're adding insult to injury when you catch spam, lower someone's income and then blog about it. At the same time, the collective value to a community that can point to your blog and say, "See, board of directors from Company X, that's why you don't want to try that tactic," is pretty high.
There's also a good amount of education to be had from exploring some of the patterns you pick up on - legitimate marketers, spammers, web developers and all will be learning.
I really can't see it as "ratting out" when A) It's your job and B) The collective value is greater than the loss of an individual page or site. The only time I think it would be truly "mean" is if you called out the individuals or group behind the work, i.e. "I know it was you Rebecca - you spammed and broke my heart!" (followed by Italian kiss of death).
That's what I was trying to say. Well written, Rand!
In the cases where the spam is so obvious and over the top is it really ratting them out? Shouldn't it almost be expected when the spam is pushed too far. And if we've identified spam as wrong then shouldn't the company responsible expect repercussions?
Matt, In cases where it's not over the top and there may be questions behind the motivation of the site you can probably still point out what the site has done wrong without actually identifying it. Detailed screen shots of specific areas of the page would still make your case without necessarily outing the company.
And if in the end you feel bad calling out a site you can always link to them. They might end up getting more traffic from you than what they would have gotten from the bad seo.
Did someone mention my name?
You know we miss you 'round here, mate. :)
I don't know much about spamming but I'd think if they were making 25k a month they should definately get into PPC or find something else to do.
As far as being called out I think most decent guys at bh or spamming count on people calling them out and I'd say it's really a waste of everyones time. Unless there's some good SEO info to point out on what they're doing correctly. No one likes a Tattle Tale!
I know some blackhat guys and the goal isn't to make one great site that ranks well, but rather 10,000-100,000 sites, some rank, some don't, It really it doesn't matter much to most of the folks I know. Being called out and making the front page on Digg is a badge of honor.
I think the common misconception amongst the non blackhat community is that "Hey I have this site called SEOMoz and I put tons and tons of hours into it, and it would sure suck if it got deleted." What you have to realize is what if you had 10,000 SeoMoz sites. Then who cares what happens to one.
The long term sustainability is definately a valid point and much like the mob the guys that are good take that dirty money and buy sites that you use everyday!
I guess that's it in one - they don't care if one getsa done because they have 99,999 other sites that do the same.
Hello, I discovered this intresting blog a while ago and I'd bookmarked it. Though not a SEO expert, far from it but always looking for ways to learn few things about SEO.
I love this part of your article, "I'm not saying SEOmoz is entirely altruistic. One of my big goals with the company is certainly to earn money. But, I'd rather have a 10% chance of making $50 million dollars 5-10 years from now with a great idea and great execution and a long-term model than a 90% chance of making $25,000 a month from spamming now, with diminishing returns in the future."
You know why this part caught my fancy? I used to be in haste to make money, but after starting out my first web site I discovered long term goals is the best, especially for quality SEO. And I am doing that now.
So I am lucky to have come across your site. I will visit here from time to time and hopefully order something here.
Thanks for your time. :)
keep in mind there is not just one set of rules...that's all I will say on that point...
C'mon Todd. You've got to give us more than that. At least a little. Maybe on your blog?
I think where you got "off topic" was actually the most interesting part. I have often had to think the same way, but i do not think that the lines of "long-terms goals vs short term goals" are as clear as let's say, the black and white of "Spy vs Spy". If they were it would be funny to see "long term goals" standing behind a bush rubbing it's hands waiting for "short term goals" to walk into it's hole in the ground trap only to end up with an anvil on his head dropped from "short term goals" smiling and piloting a helicopter hovering above.
Okay, what i mean is this. Almost everyone actually has either a "long term" goal or a "short term" goal in the beginning, but each of these extremes end up fuzzy throughout the process of creating, maintaining, and marketing their project. I find this interesting because so few people stick to their guns on this. The real enemy of everyone in this market is uncertainty. (some people, like myself, sorta get off on uncertainty) When people are scared they do dumb things, everyone (at some point), if given the possibility, would put certainty (manipulation that gets them traffic and profitability) if they were able. So I disagree where you say...
"I'm of the mind that Matt Cutts could come right out and reveal a rough take on the Google algo and really, he'd be helping Google's index quality, helping legitimate webmasters and sites to optimize and probably not helping spammers at all."
But I get what you mean.
I enjoyed that bit most too (and Rand's position seems very sensible to me - I am keen to build a valuable business as well, and am looking at lots of ways of doing that, but I agree that most spam techniques have to be short-term by their very nature).
Incidentally, feedthebot - you've become suddenly active and I'm enjoying your contribution - did you comment much before becoming a premium member? I don't remember comments from further back than a few days?
I had never been to SEOmoz before two or three days ago :)
In which case welcome many times over. You'll like it here. Everyone's friendly ;) Hope you find the premium stuff worthwhile - you're a nice 1-person stat to help Rand's conversion rates!
Like I said, I like your writing, so hope you keep entertaining us...
Thank you for the kind words!
Well I was hoping for more of a "Why is Matt being so quiet all of a sudden" undertone when I read your post title, but I do say you've got some interesting points here. I'm always for Open Source business models. In every case I feel they build companies and platforms for the better. If you have crummy code, the community can help you identify and create better logic. If you have a loophole that gets exploited, well you know about it now and your next move will only make you even stronger and more defensible. In the business models, much like you've done with opening up your financial statement, you not only provide valuable information to your industry but also allow for commentary from third-party perspective on how to improve or what to watch out for.
So I agree with you that if search engines would be more open source at their core, their business would improve from the contributions of others much more than it would hurt from the exploits of spammers. The competition getting a peek under your skirt is quite a concern though, and might be the only reason for keeping EVERYTHING closed when it comes to business intelligence. Even though we all really want a look!
I have to agree with the idea that search engines being more open would benefit them more than it would actually hurt them on the spam front. The thing with spammers, at least those at the top, is that they work on a huge scale. They don't just focus on one technique, they test hundreds of different techniques and then fine tune everything to utilize the best performing technique. They are also constantly trying new things out to stay ahead of the game. So they will always find a way to exploit the systems. By providing more info to legitmate webmasters and SEO's the engines give them a better chance to beat the spammers and they will also open up a dialiogue in which the legitimate players will be more willing to help the engines develop systems to identify new spam techniques.
It would be neat to rely on Webmasters to point these tactics out, but I think it would result in numerous debates as to what exactly counts.
Sure there's the obvious ones, but what about the ones that aren't obvious at first glance? There would have to be a mass informing by major search engines and supporters of this to get the word out about acceptible practices and a form of standard to point them out.
Personally, I would rather spend my time informing the general public about the importance of online marketing and help to clear up any myths that continue to linger around (ie. PageRank, Submissions).
I think with a big enough following, it could be done and the results would certainly be beneficial to the Webmasters who are ethical in their attempts to game the SE's. For the most part, it would probably only help SE's and hopefully would help to better the results index, like you said.
As far as I"m concerned, anything that can be done to cut down on spam would be a great thing. I agree with you Rand.
Thanks Rand...
Great information but... now I have more blogs to read.
How do you keep up with all this stuff and have time to get anything done?
I guess I am going to just bite the bullet and spend more time reading blogs on a daily basis.
I really enjoy your posts and am a Whiteboard Friday addict.
I really can't see it as "ratting out" when A) It's your job
Job? Sure a large part of Matt's job is finding and stopping spam but I think you're really reaching to say his job includes outing the spam on his blog.
btw - I love that he does it tho - great entertainment value ;)
Rand, you're absolutely right. You have the skills to write and you have the knowledge. Spaming is for desperate website owners that don't know or don't want to do any better.
You probably see the results, the number of returning visitors and paid members. I bet if these people smelled SPAM there wouldn't be half of them here.
I appreciate the quality you are putting into this blog. Hope that nobody ever convinces you otherwise.
I suppose at my age I still have that "don't be a rat" mentality kicking around back there.
I'm torn - I think calling out spammers is a good thing and can scare people in to fixing their code.
I also think it is a bad thing as it may "inspire" nefarious practices plus competitors in difficult areas could go after the competition to get them banned.
I think the balance as it is could be a good one - top bloggers with influence occasionally call out people and the SE reps sometimes call others out. On the whole though, the work focuses on improving the end user experience - not punishing those misbehaving.
I completely agree that making your own site(s)/clients' site(s) better is a much more effective way to do things than to try to find all of the ways your competitors are cheating. But having said that, has anyone heard anything about what kind of user-policed anti-spam features Wikia might be coming up with?
...seems like the natural place to report it for those who are interested in cleaning up the Internet & aren't actively going out of their way to find it.
Rand, at first I found it hard to believe that you were promoting the use of scare tactics to persude people into not promoting sites in grey ways (since you list buying links as something to out).
Then I thought to myself, Rand you are brilliant. If Matt Cutts can scare everyone else into not buying links, I can come through and mop up the $$$. I know I'm not going to stop because no matter what Google or any other SE does I'm just gonna adapt and rake in the cash while all the other people sit on the sideline scared.
Matt, can you please scare some more people for me. I'll be eternally grateful. :D
Jeremy - I'm just saying that MC can show his cards more often and it will help, rather than hurt, all of us. SEOmoz buys links for its clients - we think its a legitimate way to market a business, and an effective one. I think he should scare people away from tactics that aren't safe - if you know what you're doing, you can buy links safely and effectively. If you don't know what you're doing, you may A) have never heard of Matt Cutts and B) have never considered whether link buys (or link selling) might hurt you at the engines.