The blog post - PageRank Sculpting - from the head Google's Web Spam team is a critical read for SEOs worldwide:
So what happens when you have a page with “ten PageRank points” and ten outgoing links, and five of those links are nofollowed? Let’s leave aside the decay factor to focus on the core part of the question. Originally, the five links without nofollow would have flowed two points of PageRank each (in essence, the nofollowed links didn’t count toward the denominator when dividing PageRank by the outdegree of the page). More than a year ago, Google changed how the PageRank flows so that the five links without nofollow would flow one point of PageRank each.
It's valuable to recall the illustration I put up on Google's initial announcement of this change:
This change in Google's treatment of nofollow links comes with some very interesting additional advice/clarification:
Q: Okay, but doesn’t this encourage me to link out less? Should I turn off comments on my blog?
A: I wouldn’t recommend closing comments in an attempt to “hoard” your PageRank. In the same way that Google trusts sites less when they link to spammy sites or bad neighborhoods, parts of our system encourage links to good sites.
Many in the SEO field have long suspected that linking out to good places can provide a positive benefit, but I'm afraid that's going to be very hard to quantify and therefore difficult to justify. In all honesty, I believe we're going to see SEOs and websites revert to what I'll call "old-school" PageRank sculpting - the kind prevalent prior to the existence of nofollow.
From now on, if you wish to sculpt PageRank, you'll want to use one of the following classic PR sculpting methodologies:
- Option A: An embedded iFrame on the page containing the links you don't want the engines to follow (remember not to link to the iFrame URL, and potentially block it using robots.txt)
- Option B: Links that call a Javascript redirect script with access blocked for search engine bots (as Google is also now crawling basic javascript and counting links through it)
- Option C: An embed in Flash, Java or some other non-parseable plug-in that contains the desired links
- Option D: Settings that turn off links for non-cookied or non-logged-in visitors
Tragically, while this action won't hurt spammers or those seeking to manipulate Google, it will seriously harm many thousands of sites that have employed nofollow internally as it was long considered a best practice (and messaged as such to the SEO community by the same source as this reversal). I suspect it will be several years and many re-designs before a lot of sites are able to clean up this solution-turned-problem.
I'm saddened to say that given this change, we, as SEOs, are going to have to also recommend the best practice that comments (in all forms of UGC) no longer accept links. While Google has said that linking out to "good places" provides some value, that merely suggests that webmasters and site owners should select good resources editorially and link to them with live, followed links. Comments that contain links, unfortunately, will actively detract from a site's ability to get pages indexed (as they'll pull away link juice from the places that need it). It's likely that a plug-in for Wordpress that sends comment links out through uncrawlable Javascript or uses iFrames will emerge in the very near future.
This is a disappointing move from Google on many fronts:
- It allows malicious operators to actively hurt a site by adding nofollowed links in comments, forums and other open submission arenas.
- It removes the protection webmasters thought was afforded by nofollowing links (you may not get hurt for linking to spam or paid links directly, but you're now indirectly hurting your site's PageRank flow)
- It casts doubt on Google's credibility with future messaging to webmasters (Danny Sullivan covered this when he wrote about the loss of backwards compatibility)
While I'm personally frustrated, I'm also thankful to Google for publicly messaging this in an honest, open way. I hope that in the future, we'll get this notification in a more timely fashion. SEO consultants and in-house analysts are going to have their work cut out for them over the next few months.
BTW - Although Google has almost certainly messaged this honestly, we've got some tests running to make sure this is the case (with both the nofollow and the iframe/javascript solutions). Results will be posted here once our tests have been confirmed. We're also going to be making changes to how Linkscape's mozRank scoring system, modeled around similar intuition as PageRank, will treat nofollowed links in future indices.
p.s. Danny Sullivan's comment on Matt's blog post is also an essential read (and re-iterates many of the points above). A few valuable excerpts:
With this change, I can still get the $4 if I simply don’t allow comments. Or I show comments, but I use an iframe, so that the comment actually reside on a different page. In either case, I’m encouraged to reduce the number of links rather than let them be on the page period, nofollow regardless. If I’m worried my page won’t seem “natural” enough to Google without them, maybe I allow 5 comments through and lock them down after that.
Rather than clarify things, I feel like this is what your post is going to do -- cause people to consciously reduce the number of links they allow on their pages. We’re going to see an increase in iframe usage or other techniques to reduce links and flow more PageRank to the remaining links, for those who really worry/believe in such things.
It's been a long time since we had such a fundamental shift in SEO best practices (maybe the canonical URL tag, though it's effectiveness has been questioned and this PR sculpting reversal isn't likely to inspire confidence).
Are we missing the biggest point here from Matt:
"More than a year ago, Google changed how the PageRank flows so that the five links without nofollow would flow one point of PageRank each."
So if you didnt notice the change over a year ago, why all the fuss now? This "sculpting" people were doing 8 months ago obviously would be the same it is today...according to Cutts.
Thank you! If you're ranking well now, why make a big fuss and go change all your nofollows? If it ain't broke, don't fix it!
I second this comment. The big thing that stood out for me in Matt's post was that is all started over a year ago. With the pace that Google crawls the web, most sites would have seen the affect by now, no?
Man oh man are paginated links going to be a pain. It will be interesting to see how people deal with the various places where nofollow was previously used (links to T&C, sign in, etc). iFrames, JavaScript, Flash, etc are all more complicated to get right and can often degrade the usability, look'n'feel and accessibility of a site.
Before making any changes I'd like to be sure about the other parts of PageRank after these cahnges, in particular:
1) If you have 2 links on a page to the same page and 1 link to another page, do the target pages get 1/2 PR each or 2/3 & 1/3 PR repectivily?
2) If a page links to itself using a # does that count towrads the PageRank allocation of the page? If I had 1 link to the same page and links to 2 other pages, do the 2 other pages get 1/2 PR each (link on page doesn't count) or 1/3 PR each (in which case where did the other 1/3 go - back to the original page or evaporated?).
These may seem like dumb questions but it might be worth checking our assumptions before changing our techniques. Look forward to the result of the tests Rand.
I'd agree with the idea of it affecting users more than google realise - afterall any people using screen readers etc. are likely to be heavily affected by this.
ie: Disabled person cant find login to online banking etc, as SEO's have put login in Iframe/Javascript/Flash that screen readers etc cant read.
So whilst we may be tempted to use said techniques its going to heavily affect usability and accessibility - at a time when the EU etc is starting to implement legislation making it a legal neccisity to have full accessibility which if you resort to the above tactics could result in heavy fines & lawsuits.
Google putting us between a rock and a hard place again?
Good point Rob - I should have voted UKIP....
Well one thing we all have to keep in mind, is that if users can't utilize your site, it doesn't matter how well it ranks. SEO's have to consider usability right alongside ranking well. If you're doing things that cause usability issues then you're focus is all wrong. (and by you I mean SEO's in general)
I've been wondering the same thing. My site currently has three links to the homepage on every page
Does this help steer the flow of juice back to my homepage?
I could easily add at least two other links back to my homepage without detracting from the user experience:
Only the first instance of a link to any individual page is counted, but whather they are discounted totally or whether further instances of links to that page would be utterly ignored or whether they would dilute PR in the way which nofollows now are, I could not say.
Stephen, Those aren't dumb questions at all. Those are 2 of the questions that every SEO wonders about, as soon as he/she takes the time to really understand PageRank. I don't have the answers, but I'll add my thoughts. If the "random surfer" analogy holds true, it would seem logical to factor all links into the PR calculation. In other words (referring to your #1 example here), Google should distribute twice as much PR to the first page, since a random surfer would have twice the probability of clicking on a link to that page. However, keeping track of multiple links per page and using that data in the PR calculation seems like it would require a significant increase in programming, storage, and computational resources... which leads me to believe that Google does NOT track multiple links per page (at least not when Google was first released). Also, if you read section 4 of the original Google paper, it says this:The URLresolver reads the anchors file and converts relative URLs into absolute URLs and in turn into docIDs. It puts the anchor text into the forward index, associated with the docID that the anchor points to. It also generates a database of links which are pairs of docIDs. The links database is used to compute PageRanks for all the documents. To me, this sounds like it's possible to get credit for multiple links with regards to anchor text... but not PageRank. Again, that's just my opinion, and certainly things may have changed since that paper was written. Basically, I assume that Google calculates an on-page probability that factors in things like {relative font size of anchor text} and {location of anchor text within a document}. Using a complicated model like that would allow multiple prominent links to distribute more PR to a given URL--even if it might not be as simple as "twice the links = twice the PR." (Damn... I just reread my comment and it did absolutely nothing for answering your questions. Oh well. I tried. Please don't hate me, Stephen.)
I'm just glad you didn't think my question was dumb. Quite a compliment coming from you! Thanks for sharing your thoughts on it. I think you could be right, the question is, what experiment would you run to prove it?
I'm not usually one to freak out over Google changes, but I think what really stings about this one is that nofollow is a tool Google explicitly gave us and endorsed, and they created it for some perfectly viable reasons. Now, after endorsing it and convincing us all to use it, they changed the rules a year ago without any warning at all.
When Google makes a set of secret rules in the algorithm, we try to divine those rules, and then they change them, that's one thing - it's the game, and SEOs have to play it. When Google says "This is what you should do" from the mountaintop and many of us do it to try to play by the rules (we're not all siloing for maximum gain), and then they secretly change those rules, then the message I personally start to hear is "Stop doing what we tell you, because we may just screw with you anyway".
The next time Google says we should do things a certain way, why should we listen? This is a bad trend and the Google team has potentially lost a lot of credibility with the white-hat community, IMO.
They endorsed it for blocking comment spam mostly and I think we as a industry help morph it into this sculpting thing. Cutts has always said it would not be the first thing to worry about. But then all "SEO tests" came out and said it did help pages rank, without any real proof. Michael Martinez has been posting that you cant control Page Rank in this sculpting fashion for well over a year, with specific comentson many of these blogs, but none would listen.
So many bought into it when they should not of. Good post by Aaron Wall on this "testing" today:
https://www.seobook.com/worthless-hype
Practically speaking, I haven't used nofollow as a crutch and I'll survive, but I do think that it puts a serious dent in any future tactical endorsements Google makes. Even if it was just for comment spam, that still begs the question: what do we do now? Many people invested time and energy in custom code, plug-ins, etc. to use nofollow to combat comment spam and help Google, and now Google says "Sorry, try again".
Sure, plenty of people gamed the system, and I know Google has to contend with that, but what should the well-meaning webmaster who was trying to help Google fight spam think? That person may not listen to Google the next time, and I can't blame them.
agreed Pete - doesnt matter whether this was around nofollow or something else - the principle remains the same. How as whitehat people are we supposed to keep faith in being whitehat and following the rules when they are changed arbitrarily like this?
I think you need to read the article again and any previous articles related to the subject, for one you have been able to sculpt page rank with the nofollow attribute and internal link architecture, until recently Matt Cutts did say it was ok to do so except it was not something to put much weight on.
Um I thikn you might need to re-read it, Cutts said this change took place a year ago so this PR sculpting (if it ever existed) stop over a year ago.
incrediblehelp I'm not arguing the fact of "when" the algo changed to address nofollow pagerank sculpting but that Matt did in fact say "you could". https://searchengineland.com/pagerank-sculpting-is-dead-long-live-pagerank-sculpting-21102 So going on and on about not being able to sculpt page rank from day one and act as if "it never existed" is wrong.
Well that is where I differ, I have never seen tests where websites ranked better and the success can be COMLPETELY attributted PR Sculpting.
Sure mnay SEO's have said they have seen those results, but the fact is their is far to amny other variables ot say that this one made that big of difference in traffic and rankings.
incrediblehelp, Google is never going to endorse a practice that focuses solely on improving rankings. Everything you ever ask Google will be answered in a way that promotes an improved user experience. The SEO community wasn't interested in whether or not Google "endorsed" PageRank sculpting with nofollow--it was interested in (1) does it work? and (2) are we allowed to do it without risking penalty? The answers we originally received were (1) yes, it works, and (2) yes, you're allowed to do it. The only thing that has changed is whether or the nofollow attribute is an effective tool for PageRank sculpting. We're now hearing that no, it is NOT an effective tool. However, Matt Cutts continues to inform us that yes, PageRank sculpting works, and yes, we're still allowed to do it. These answers are as recent as 3 weeks ago, when he posted this video: What are your views on PageRank sculpting? Summary PR sculpting: Does it work? YES Is it unethical? NO PR sculpting with nofollow: Does it work? NO Is it unethical? NO And one last thing I'll add... For God's sake, don't cite Michael Martinez like he's some kind of expert. He only comments on this blog in hopes that his contrarian perspective will confuse people into subscribing to his own blog. The only "evidence" he's provided that supports his anti-PageRank-sculpting philosophy is the following logical fallacy: "Until you can measure it you cannot sculpt it." I mean come on... is it really that difficult to understand why that statement is completely meaningless and invalid? Even a broken record lines up with the music at some point. That doesn't mean you should dance to it. (Dude... that was deep.)
LOL, I'll second that.
Yeah really. I guess this is why testing is so important in SEO. But in reality..who has the time or budget available to test everything G says?
It says one thing to me - Google feels much more confident in distinguishing between the quality of links. Maybe, it's time to abandon the nofollow at all (even for external links) and let search engines to decide which links are good and which are not.
But don't you-webmasters-SEOs feel stupid? We helped Google by nofollowing links to not count "wrong" links, because they couldn't do it on their own. Now when they feel confident, it turns out, it can even hurt your site.
Personally, I am not going to change anything on my websites for months, maybe never. But, of course, I am very interested in the test you run.
Have to admit, Jill Whalen has been saying this all along.
IMHO - adding IFrames will not be a long term solution. It is impossible to hide the target URL of the IFrame and, as webmasters add them to their pages, Google has no reasons not to consider content of those IFrames as part of the "regular" content on particular page. Eventually links in IFrames will be counted as links on page.
Do you think Google would disobey robots.txt to reach those URLs? Seems somewhat unlikely to me.
From point of view of search engineer I see no reason not to consider it as "regular" content on page even if it is framed or else why would you want to show it to your visitors? It might not be indexed as standalone URL because of settings in robots.txt but I truly believe frames might get flattened in a similar manner how graphical editors flatten image layers while exporting to JPG for example.
Why shouldn't it be done? Search engines want to see the same content that end user see on the site and this is just next step.
In order to "flatten" the original content and the iframe content into a single document, Google would first have to get the iframe content from your server. If you have the iframe URL DISALLOWED in your robots.txt file, Google cannot access it... unless googlebot makes an intentional decision to disobey the robots exclusion protocol. In other words, to accomplish what you're suggesting... Google would have to spit in the faces of webmasters everywhere (not to mention, they would have to change this or admit it was all a lie).
But then isn't having an IFRAME to a page blocked in robots.txt a pretty good indicator for Google that you're being naughty (not a perfect indicator).
Plus, on the subject, how does Google discover cloaking? Unless they disobey robots.txt then they can't do it programmatically. That means that for most of the 200million websites cloaking wouldn't be found??
Maybe I've got a bunch of tinfoil on my head, but can't Google opt out of the REP the same way as they just did to nofollow?
In the words of Jack Sparrow, "the only rules that really matter are these: what a man can do and what a man can't do."
Google's much the same.
Why they disobey robots.txt ALL OF THE TIME
Examples?
Cant show as it is a clients site, but I doubt it is on purpose. Usually happens after pages are indexed and you are trying to block them through the robots.txt file after the fact. The pages still get crawled and indexed. of course this is because the pages probably have links pointing to the pages, but still it should be blocked because of the robots.txt rule right?
That will not happen if you use the "noindex" directive in your robots.txt instead of the "disallow".
Or add the meta noindex to the page you want excluded.
yep meta nofollow,noindex on your clients links/resources pages...
You should use the meta noindex, but not the meta nofollow.
Just remember, if it's on robots.txt they'll never 'see' the meta noindex.
once in the index you'd have to use the webmaster tools to remove the page - thats what its their for, you shouldnt be using robots.txt to do that as its bolting the door after the horse has bolted.
or however that saying goes... :D
Here's a quote from Matt Cutt's blog:
Everfluxx, a page that is blocked by robots.txt can still accrue PageRank. In the old days, ebay.com blocked Google in robots.txt, but we still wanted to be able to return ebay.com for the query [ebay], so uncrawled urls can accumulate PageRank and be shown in our search results.
https://www.mattcutts.com/blog/pagerank-sculpting/#comments
Straight fom the horses mouth:
"Everfluxx, a page that is blocked by robots.txt can still accrue PageRank. In the old days, ebay.com blocked Google in robots.txt, but we still wanted to be able to return ebay.com for the query [ebay]"
https://www.mattcutts.com/blog/pagerank-sculpting/#comment-348361
That is correct. If external sites link to a page that is disallowed via robots.txt accrue PR.
Yes acrue PR and then the pages will show in the search results, but my robots.txt is telling them not to! That is my whole point.
Why are you doing that Google?
Yes acrue PR and then the pages will show in the search results, but my robots.txt is telling them not to! That is my whole point.
Why are you doing that Google?
OFF-TOPIC: Sorry. Duplicated post.
If you use the "noindex" robots.txt directive instead of the "disallow", those pages will not show up in Googles index. That is for sure. I know it is unofficially supported by Google, but I still use it, and that for over a year without any problems. Basically I use it as one of my best tools for bots herding.
incrediblehelp, THAT is your basis for claiming that Google disobeys the robots.txt protocol?! I'm sorry, but that is just ignorant. Before posting such bold claims, you might want to take the time to actually read the protocol in the first place. The original robots.txt specifications were written before Google existed, so it was obviously not written with PageRank or Google's index in mind. The point of robots.txt is to define which parts of a website are accessible to bots. Googlebot would be disobeying robots.txt if it requested disallowed resources from a server. There's nothing that says Google can't return disallowed URLs in its search results. Google will display as much information about the page as possible, without actually fetching the page and crawling its content. Usually, that means it shows only a URL, but it can also use the OPD or possibly even archived content that was fetched before the robots.txt file was in place. I'm now going to reread every comment you've made on this post and thumb down any that I disagree with or don't like. Cheers!
The iframe itself might be counted as a link. I'm sure it doesn't pass pagerank but might be considered as a link. If so, creating an iframe for a single link would be the same as just nofollowing a the link.
Is this going to lead to creating iFrames inside of Flash files located in directories that are disallowed through a Robots.txt?
Bother! Now dust off my Flash MX.
All joking aside, I have used equally-extreme methods. For example: 1. A site has a global navigation element that links to several un-important pages that shouldn't be ranking or accumulating PageRank. 2. I remove the global navigation element code from each page and put it in an external file (the iframe source URL). 3. I add this Meta tag to the iframe file: <meta name="robots" content="none" /> 4. I define a JavaScript function (in another external file) that writes the iframe element code into my original document. 5. I disallow the external JavaScript file in robots.txt. 6. I leave the un-important links on one or two pages (e.g. sitemap.html). This lets the search engines crawl those un-important pages, but without giving them more PageRank than they deserve. This also gives users access to those pages, even if they have JavaScript disabled. 7. Each page that used to have an entire block of code (of un-important links), now has just a single JavaScript function call, enclosed in <script> tags. 8. You could even take it one step further and define your JavaScript function to write the iframe element's code into a specific element ID (or using the DOM). This would eliminate the need for <script> tags, except for the ones in the <head> section that reference the external .js file. The irony is... I'm obsessively white-hat. I'm not "hiding" anything or doing anything sneaky. I go through all that trouble just to make sure Google doesn't mistake my Privacy Policy for a landing page.
About your 5th point: "I disallow the external JavaScript file in robots.txt."
You might not have bad intentions, but myself and Matt Cutts would advise you not to do that: https://www.youtube.com/watch?v=PNEipHjsEPU&feature=related
Ha ha! Did you see who sent in that question? ;)
Well why did you say that here if you already knew that it is wrong? :)
I understood his answer a little differently. I don't disallow Google because I'm trying to hide "sneaky redirects" or anything. Most of the time, I disallow CSS because I want to make sure Googlebot is using my HTML tags to determine on-page importance... and NOT trying to interpret my CSS styles.
I never thought that you are trying to do something against Google guidelines. But we both know that bots can misinterpretate some stuff we implement, even if they are not violating any rules.
I you fear to use the noindex robots.txt directive because it is still not supported officially by Google, you still can use X-Robots.
If you are on Apache, you simply need to add these lines in your .htaccess, and you will achieve what you mentioned above without the slightest problem.
<FilesMatch "\.(css|js|txt)$">Header set X-Robots-Tag "noindex,follow,noarchive,nosnippet"</FilesMatch>
"Tragically, while this action won't hurt spammers or those seeking to manipulate Google, it will seriously harm many thousands of sites that have employed nofollow internally as it was long considered a best practice (and messaged as such to the SEO community by the same source as this reversal). I suspect it will be several years and many re-designs before a lot of sites are able to clean up this solution-turned-problem."
Matt's post said that this has been in place since last year, therefore surely this won't be the case? If anything it would have been an issue whenever this was implemented but the change supposedely wasn't picked up by anyone. I don't think we're suddenly going to see drastic drops in traffic as a result of this announcement.
We are talking about Google here... But does this also involve Yahoo and Bing? Do we need to use different techniques for different Search Engines now?
yahoo and bing are still ... irrelevant
not true - I read Bing's search share was up to 10%! (sarcasm)
I'm sure Google is shaking in their boots.
Option E - Increase the amount of internal linking and flatten site architecture.
My old Sandcastles linking structure works great with the new algos, though there is now a need to remove external links totally from dupicate content pages rather than nofollow them.
Wordpress does this by default with their really ugly automatic snippets
Option F - there is an even better way, that maximises the benefit of user generated content, still providing dofollow links, but retaining 95%+ of the juice from all external links on a page, without using nofollow at all.
@AndyBeard I have found having a flat site architecture can often hold back or reduce the potential size of your site if you dont plan carefully when you are building the initial architecture...
Did Google just do this to see how many sites would dedicate resources to removing the PR sculpting and flag who is abusing the use of nofollow links purely for SERPs and not for user benefit.
Did they play a good game of simon says all those with PR sculpting raise your hand to be manually audited...?
Andy - Option E - are you suggesting linking to a page on the site that detects the user's click location and redirects them to the right destination somehow but then shows links back to the rest of the site on that redirection URL for engines? I'm a little confused, but if my interpretation's right, it sounds potentially manipulative.
Rand, and what do you think about using link www.domain.tld/#page-xyz instead of www.domain.tld/page-xyz with "nofollow" Google says everything after # sign doesn't count. Maybe it is not exactly the same, but at least you get an extra link to the home page instead of linking to page you don't want to link.
Makes sense... And then you can just use javascript to process and redirect that click. We'll have to test that out - thanks Pipuks!
If Google follows a link and ends up on a different page, that is cloaking, though "link cloaking" not page cloaking.
Considering Google place a value on links, it might be a dangerous concept.
I have considered dynamically changing links on a page depending on source of traffic mainly for PPC.
Example (for E)
If you currently have 5 internal links on a page and 5 external links
Increase that to 50 internal links and5 external links
In my Sandcastles approach I used dynamically increasing number of tag links depending o the number of dofollow comment and trackback links.
The old code is broken due to WordPress and my clumsy editing, but needs a refresh due to changes in WordPress tagging anyway (I used to use UTW)
At the same time the tag pages now need to be much higher quality, and have external links removed rather than nofollowed.
The same effect can be created in many different ways, especially on social sites where you can have links to user profile pages that have lots of internal linking.
@TheLostAgency
There are possibilities of a negative effect, but the key is to have more linking on pages that have lots of external links, but on pages that don't have lots of external links, you can then use laser focus of juice and anchor text
Edit: the bold external previously read internal
I also wonder if this is a precursor to starting to penalise excessive use of nofollow. I read into Matt's comments a note of "you have used and abused this too much". By starting to tell people not to use it as much, they can then, as a next step, consider penalties based on large-scale use... The first step in wiki's downfall?
Not sure about the "wiki's downfall" theory. Wikipedia doesn't excessively use nofollows - only on external links, which are usually only a small proportion of total links on most pages.
I could see the penalty scenario happening for sites where every link except the sales letter page is nofollowed - but what would be the point in light of this announcement? By negating the tecnhique they will eliminate it without the need for a new penalty - a case of "its pointless" rather than "its manipulative"...
how that might play out with twitter, facebook, etc. social media sites that seem to nofollow across the board?
@will I think it might be, but only as it applies to internal link sculpting.
I was advised strongly to use the "nofollow" as it applies to UGC and outbound links by a Google engineer.
I don't see it having an adverse effect on the social media sites or forums.
@Rand Thanks for the verification on this.
I would love to see this announcement be a precursor to Wikipedia dropping the nofollow from its references. Having a nofollow there in the first place is pretty rediculous since all of the content on their domain is attributable to said sources. It's like they're saying, don't trust these external links, but do trust our website even though all of its content comes from those external links.
I really don't think Google will be punishing "manipulators" as apparently it's been over a year since their "manipulation" had any effect at all...
"It allows malicious operators to actively hurt a site by adding nofollowed links in comments, forums and other open submission arenas."
Yep, comment spam just took on a whole new meaning. Man, I can almost see a small army of "content writers" in Bangalore feverishly typing away in their little cubicles. This can't be good!
For me the main negative of all of this, and one Rand referred to in an earlier post, is that all the X-File fans who think that Google is Skyney and Matt Cutts is Arnie, are going to be able to say "See, we told you!"
Evaporation is an interesting term. It has an implication of a slow, unseen process.
To my understanding, one day (or short timeframe) last year Google changed how much PR got passed.
One day all the links in the world passed X pR. The next they passed X-Y PR. And no one noticed?
The mechanism Matt Cutts is talking about doesnt seem to match the perceived outcome.
Matt says its evaporation, but what he describes should have been a big bang.
That big bang should have removed a hell of a lot of PR from the system in one short time scale. Pages should have been removed from the index, PR should have dropped etc etc.
I am struggling to believe all you bright folks would have missed this cataclysmic event. There must be something else that has not been communicated by Google that fills in the gaps.
I did notice loss in PR across all my sits about a year ago. I couldn't figure it out...but now it's all making sense.
There's one thing I still can't understand. What happens to the PR that isn't passed when you have a nofollow link. Does it stay on the page with the nofollow link or does it just evaporate?
As Rand correctly points out, this change suggests a return to "old school sculpting" -- what I originally labeled Dynamic Linking -- and the ebook still contains the original Javascript-based code.
But IF there really was a change in nofollow and IF the math of the example is being correctly described, then there is a "silver lining" to this change for all by the web's largest sites. Here's why.
PageRank is a probability distribution and the totality of PageRank across the entire Google index has to sum to 1, so the PageRank missing from this change does not simply "evaporate" -- it is added to the "random teleport" probability on the page where the nofollowed links appear. This is a random PageRank bleed!
At Dan Thies' request I ran the simulation in detail with a hacked version of the PageRank code to include this change.
It is the largest sites making the most of nofollow sculpting that will be bleeding the most and the bleeded PageRank feeds the rest of the Google index. Makes absolutely no sense.
In the comments section of Matt's post, he mentions how you wouldn't be too far off if you imagined that this PageRank was going to the reset vector, so your results of your original hack may not be far off if this information is to be trusted.
There is one interesting thing to note with the "simulation" you did. Even with that PageRank bleed, your 0 page still ended up with ~.2 PageRank points more than it had without the sculpting.
So in essence, perhaps you now have a very definite point of diminishing returns that will penalize for aggressive nofollow use but still provide benefit with light use.
Of course when we get out to the "real world" it's not quite so cut and dry when we don't have Google's PageRank numbers and we start adding all sorts of inbound and outbound links and so forth, but it would stand to reason that a very light use of nofollow would still provide benefit over not using it.
However if the above is true, I would prefer to look into other methods (external java, overhead page consolidation with name anchors, and taking more time to really think out site structure).
According to Cutts, the computations are way more convoluted than what we know based on the original PageRank patent. He also hinted that the lost PageRank is "reset", not redistributed to the rest of the web - whatever that means. So I'm not convinced that nofollows result in random pagerank bleeds.
He says that what happens to it is essentially what happens to the reset vector - the 15% (in the original PageRank paper) probability that the user requests a new page.
Here's what he says about the reset vector in his PageRank sculpting post - "you could think of it as 10-15% of the PageRank on any given page disappearing before the PageRank flows along the outlinks."
But as Leslie mentioned, probability values don't really "disappear." The reset vector would distribute PageRank across the index, as the user would be requesting a new random page.
So if what happens to the PageRank of a nofollowed link is essentially the same thing as what happens to the reset vector, then he's saying that it would be distributed back across the index.
Of course this is the first time I've really considered PageRank at this level of detail, but I think I got everything right. :)
It sounds to me like Matt there's just referring to the decay factor.
"he's saying that it would be distributed back across the index."
No, I asked him this
"So what happens to the PageRank that belongs to those nofollowed links? For example you have a page with 50 “points” of PageRank, 50 links, and 25 of them are nofollow. So that page passes 25 points of PageRank. What happens to the other 25? Does it get discarded? Redistributed to the rest of the web?"
His answer:
"Halfdeck, it’s a bit complicated, esp. since Google doesn’t view pages exactly in the framework as “classic PageRank” any more. You can think of that PageRank going into the reset vector without being too far off."
Nevermind. I didn't really digest your whole comment. My bad. Yeah it does sound like he is implying that "lost" PageRank gets redistributed across the entire index.
So in essence, by having nofollow links, you have a chance of distributing that PR leak to your competitors?
I see a tagline coming... nofollow your competitors to Top10.
Google changed this a year ago, yet apparently no one noticed. In light of this, what analytics are valid for SEO? How granular and specific can they be?
I thumbed this down. Why? Google givith and Google taketh away.
Any SEO who put true value in "sculpting" through no-follow was simply taking the easy route of fixing their site architecture.
I know I am probably going to get flamed for this, but watching professionals throw their toys out of their cot because daddy took away their rattle is a sorry sight (yes you too Rand).
If you were using'nofollow' as a signal to the search engines for it's intended purpose then you have nothing to worry about, and nothing to change.
All of this scurrying around to find alternatives is going to be a short-term fruitless exercise.
There is not now, nor has there ever been, any visitor benefit in employing nofollow. It has always been a manipulation of the Google PR game. Period.
Google claims that sites built to be user-friendly and that incorporate great content don't have to play "SEO games". Yet they simultaneously built a system that encouraged and rewarded the use of nofollow to prevent the flow of PageRank from one page to another. Hypocrisy? Hmm...you be the judge.
Regardless of the intent of using nofollow - whether related to paid links or simply in an effort to preserve PR - this practice has absolutely no benefit whatsoever for a site's visitors. It is a major reason that SEOs with any amount of experience know that when Google says, "Don't worry about the nuances of our algorithm, just focus on building a user-friendly and content heavy site...", they're full of crap.
There is not now, nor has there ever been, any visitor benefit in employing nofollow. It has always been a manipulation of the Google PR game. Period. I disagree... if only slightly. I'm certainly not an average visitor, so I do acknowledge that the following exception is an irrelevant extreme case, but nevertheless... it's an exception: I have a customized stylesheet for Firefox that formats nofollowed links (including image links) differently than the rest. This allows me to easily spot all the links that a website has chosen to "not vouch for." This may seem like trivial information to most people, but it actually plays a significant role in my browsing decisions. For example, it suggests: Which pages does the site consider un-important? Which links point to independent sites (vs. subdomains, parent sites, etc.)? How far have the site owners gone in their efforts to SEO the site? Which links are paid links or advertisements?
"There is not now, nor has there ever been, any visitor benefit in employing nofollow."
You're kidding right? An enormous reduction in blog comment spam and wiki manipulation (just to name two instances), is not a visitor benefit?
I assume she is referring to the direct/instantaneous benefits.
You're correct, Darren. I was referring to "direct/instantaneous" benefits to the average visitor while actually on an individual site - not the effect of nofollow on PR flow to other sites and potential impact of that on SERPs. There are always exceptions, and - as you pointed out - you're certainly not "the average visitor".
The vast and overwhelming majority of visitors don't even know of the existence of nofollow links, nor would they want to. Google encouraging webmasters to employ nofollow knowing that the primary intent in doing so is to manipulate algorithms flies in the face of everything they say about building sites for visitors, not for search engines.
It may have helped them do a better job of cleaning up SERPs, but it did nothing to improve the experience of someone visiting the site that employed the nofollow.
For some years we've been fans of "PageRank massaging" rather than "PageRank sculpting"...where we add a subtle "targeted link text paragraph" at the bottom on many pages, focussing text links back to say five pivotal pages of the site...with exceptional results, one of which is a remarkably high number of double/indented listings.
Doesn't always suit every site/brand/positioning but for a big site of 10k indexed pages, it really does provide uplift.
Option C: An embed in Flash, Java or some other non-parseable plug-in that contains the desired links
Just wanted to point out that Google is also parsing flash...great idea to use the iframe. However, I think the best solution is just to have great content and enough inbound links that it just doesn't matter that much if any link juice is lost.
I was wondering about multiple links to the same destination url.
Does having a link in the Menu, a link in the Nav bar and a link in the body content all pointing to the same destination count as three links?
It must have been past my bed time when this hit the site. (late to the party)
Personally, I think Darren might be right in that all those standing will now be marked as "manipulative little beaches."
Face it people, this just makes us more valuable. Job security for keeping up on the trends baby.
Why redo Google's maths? Useless. Will fail.
Think outside the box. Just feed Googlebot with another HTML element when a link is condomized.
I've posted sample code on my blog:
https://google.com/search?hl=en&q=vaporize+google
(Not sure a direct link to this rant is appreciated.)
This is definitely an interesting one. I think some re-reading of Matt's less than crystal clear post and Danny Sullivans respone are in order.
One the surface of it though my spidey sense seems to be saying this is a confusing and potenitally harmful change. From the comments on Matt's blog it's clear that he has already confused a lot of webmasters/website operators as well as riled/confused a lot of SEOs.
I believe if Google wants SEOs to help them index relevant pages and bypass irrelevant pages through nofollow, there needs to be more of a tangible incentive for us. Hence the control of link juice was perfect.
Thanks Rand.. would be great to see the results of the tests you mention.. wow what a change for SEO.. the industry remains anything but boring.
I guess very careful attention to site structure and hierarchy is required here, its going to push SEO's to demand some significant site design changes and we'll have to be very carefulwe dont harm site usability in the process.
What about the idea that the first link on a page is worth more than the last, therefore sculpting the last link on the page will cause less of a detriment.
This is particularly important in the case of your UGC (read: comments) link
Unfortunately Rand, I agree that this will probably lead to even more complex operations with how sites handle all the various kinds of links that a page might acquire.
Even more unfortunate, it may be a necessity to help protect the integrity of the page or site, such as the case of excessive links being added via UGC/comments to force dilution. Then again, for this to be effective it would probably have to be enmasse, and then not much different from regular comment spam.
The troubling part is that, aside from outgoing links in comments, no one had to adopt the idea of PageRank sculpting unless they wanted to....it simply could have been a tool to be used. This change seems to force a more active consideration on all sites.
I can't help but think there is a much bigger issue at hand. I always had concerns about users applying nofollows in any kind of sculpting manner without thought or planning since a website is like an organism, all interconnecting....whose to say what the greater impact might be to downstream PageRank flow from blocking off section. So many of us made careful, thought out decisions on usage, but still challenging for anyone to calculate impact to a site, let alone the entire linkgraph.
And perhaps that is the greater issue at hand, the larger impact to the linkgraph and any rippling effect that all this additional nofollow usage has had across the web. The fact that this change was made a year ago and is just being made known, while all the time Google representatives were making, albeit sometimes reserved or cautious, recommendations on the usage. I respect Cutts & White and the team at Google and feel they may be feeling a little caught in the middle here, more messengers than architects.
Phew... was starting to think 2009 might be a boring year for search.
Guys, Personally I think there is a lot of over-analysis going on here. As Matt stated on his post, is focusing on page rank sculpting really a valuable use of our time?
I look forward to seeing your test results Rand - the point you brought up about additional nofollowed comments on a page hampering page rank is an interesting one.
I can imagine spammers in competitive niches adding endless nofollow comments onto high ranking sites.
Rand, I read over WebProNews that "this change has been implemented since one year ago but no body notice" as claimed in the article so I want to say:
Happy Birthday New-Google-PageRank
I haven't read the other comments so I apologize for any duplicity.
What are your thoughts on how this may impact overall domain juice?
Also - I kinda get the feeling that Google may be attempting to put a cap of some sort on the link building efforts of SEOs.
Is this the beginning of a move away from link value being so prominent?
If there are two no followed links on a page are those two "points" just kept within the given page or are they lost?
Good question Rob, this is exactly what I want to know.
There are different alternatives mentioned here, how to hide links, like for example iframes and javascript. Why not hiding links from the bots with a server side (PHP) script in combination with some .htaccess rules?
There is a big difference between those mentioned and server-side solutions. With JavaScript, iframes, and the nofollow attribute, you're not showing different content to different users. You're simply exploiting the search engines' inability to crawl certain kinds of links. The risks of server-side techniques are much more serious, as you soon find yourself in cloaking territory.
Darren it is not about showing different content to different users. It about serving users content that is not served to search engines at all.
Google says that it is a violation if you are serving different content to search engines than to users. So where is the violation?
It depends on the implementation. Did you have something specific in mind? You mentioned .htaccess, so I'm guessing you're talking about something like this: 1. Server receives a request for an un-important page*. 2. Server determines if client is Google. 3. If not Google, return the resource (200 OK). 4. If Google, return a 301 Permanent Redirect to an important page (e.g. the home page). *Un-important page = a page we don't want to accrue PageRank. This example can be interpreted in different ways. You might interpret it as "serving users content that is not served to search engines at all." However, a Google employee might interpret it as "serving a body of text to users, but serving a redirect location to search engines." Other implementations are possible, but I can't think of any that would retain 100% PageRank AND avoid violating the Quality Guidelines.
I have to disagree a few previous comments. If your site (or client) was in trouble with duplicate content, I think the proper use of Rel=canonical tags can help with actual pr leakage/duplicate content and therefore sub-index issues as one sees in the G. webmaster acount or site:example.com and analytics, and this can be proven if you have the experience.
Also the way it is described by Cutts, the no follow tag does seem to add to leakage or rather 'evaporation' of linkjuice and as yet test results have yet to creep up and yell BLAMO into your face -we can infer that by having tones of comment links on your site/forum/blog...You will need to look out for your own site, and remove/handle the user comments' with some care.
I'de say remove the standard url field... still accept comments, keep your forum open for links; as this is how you drive traffic to your site keep the lights on.
I have seen a site get well over 10k longtail terms so specific to the niche they dominate many topics. Yet it still goes without saying that forum creates traffic and link backs that all make it to the main page in the end.
If you do not have these components, pr sculpting is more of an art of keeping architecture on point over internal/external/inbound links as it always has been. Certainly linking to authorities/sources is sometimes necessary and according to Cutts, somewhat welcomed.
Matt C. himself explains he rel=nofollows for conflict of interest reasons and to keep his rss out of the serps (both still endorsements/payola). So with this I think we get the point. Use of rel=nofollow stops the endorsement according to the Goog, but in reality we all see who we link to and who is linking to what brand.
Matt is just asking we all be stewards of our links, so he himself does not have to control all spam linking, meanwhile as mentioned there are ways around not using the nofollow tag - it is a non endorsment link with costs associated to the linker and no benefit to linkee.
Good luck all.
So, then, what's the best practice about Privacy Policy/Terms of Use/Contact us/etc links? Certainly you don't want to hide them completely from SEs, because you do want to show that you have them.
But you don't want to dillute the linkjuice you might have either!
The main thing that would work is increasing the total amount of links on a page, but that seems to be the completely unnatural way to go about it.
I'm going to basically repeat what Case brought up because I don't see an answer.
Let's say you have an image that is linked and a text link beneath it both going to the same URL. Previously, I would have said to nofollow the image link to give the text link the credit. Sounds to me like now, the better option is not to link the image at all or to turn that link into a format that Google doesn't see as a link (Flash, external javascript, etc.).
Am I understanding this correctly? Thoughts?
So tell me this, I remember running a site a few years back and granted things have changed but never the less, I was able to gain a PR of 1 within a few short weeks and now on a different site with a whole lot more quality and unique content, I can't seem to break the basic barrier of a 0 PR. I understand that there are different niches but it seems to me that something is terribly wrong.
When I do a back link check, I only have one link showing back from my YouTube channel, and a couple of months ago I had five or six showing up in Google's backlinks checker, then slowly they would disappear and now I am down to one link. I have backlinks from PR4 and PR5 sites, some from indexes and other writing recommendation posts with only my link there. None of them show up. And while I understand that Google may have changed their PR/link calculating algo to reflect the low PR link backs, it seems odd to me that a PR0 YouTube link would count for something while a PR4 or PR5 established site linking out would not.
Here's my address, keep in mind I have recently (as of this comment) moved to blogger and quite possibly lost SERP positions being that my host dissipated into thin air and I was unable to access my Cpanel to set 301 redirects and so I have unfortunately lost standings and traffic which I have been building up for months. I also hope Google doesn't penalize me for seemingly duplicate content because I reposted some of my older posts onto Blogger.com (still under my domain though) but now they are listed as all under one date. The quick fix would be to use the same permalink structure but unfortunately Blogger.com does not allow for changes in the link/permalink structure and so I am now stuck.
I know, why didn't you just find another host, well it's a long explanation and I've said enough. If anyone cares to add their two cents, I would certainly be very grateful to you and your help.
Westminster College Blog
Vitaliy
Hey Rand, I have just read through the whole comments thread, as usual their are a lot of interesting views on the subject.
Did you guys ever get the results back from the tests you mentioned? I think that would help clear up a lot of the speculation going around and also set out some guidelines that we can all work along. A few of the people I have been speaking too claim that PR sculpting using nofollow is still working really well for them.
I'm currently working on a tube/video site that has 230k inbound links but around 70% of links on the homepage are latest/featured videos, and then around another 15-20% of links are too external sites. From everything I've read it looks like the only solution is to iframe these links and increase the number of links on the homepage to the main categories we're trying to rank for. Anyone got any suggestions/alternatives on this?
Google say that Page Rank is not the most important factor to rank well on the SERPs, But in my experience the pages with most PR on my site (Free Android and Iphone games), rank higher than the pages with fewer PR. Thanks for the explanation of PR.
Hello Rand,
I had my blog dofollow previously, but due to a large number of spam comments daily, I added nofollow to all comments and OBL's on my site. Will it improve or degrade my site's rankings?
and......Should I nofollow About me and Contact me Links.
Waiting for your reply....
Thanks.
5 Years on, having arrived here via Google still trying to comprehend this one...
Hasn't Matt Cutts already stated that Google uses a method along these lines? True, he says the Google algorithm for calculating it is more advanced than the model that Rand is discussing suggests, but that still suggests that the old pagerank model is still part of their model. Similarly, the fact that Google has stopped people from using nofollow in this way would seem to suggest that it was influencing the Google results (why else would they discontinue it?), which means that pagerank sculpting using nofollow probably did work at one point
While Michael is right that we don't have a perfectly accurate model of how pagerank flows, that doesn't mean that we can't use past information to deduce that it does flow. It just means that, like any other method used in SEO, it should be augmented by other methods which give a good user experience. Sure, reducing the number of pages on a site will most likely influence page rank, it always would have, but since we can't measure how much it will influence it will have we should be wary of putting too much focus on it, especially if it could be badly applied and degrade the user experience
Food for thought - Option E: HTTP POST
Adding a POST form with a single submit button that redirects to the target of the anchor.
I'm not condoning this method, but it seems to work
Thoughts?
I've set up my robots.txt to disallow some javascript files that were causing errors when google would index my site and I'm still getting those error... Is google disobeying my robots.txt file?
I wonder if once they index a file and have it they'll continue to crawl it but if you disallow it from the get go they'll never get to it?
Or, do I have a unique situation where my sites reporting functionality, which tells me any error that happens on my site along with the ip of the user that had the error so I'm able to know that google is continually trying to crawl a file I disallowed?
Can the googlebot cache files?
Google definitely caches files in many different ways, but I don't know if they continue to access those cached files after the live version has been disallowed. However, according to the robots.txt protocol, there is nothing that says Google would be obligated to throw out their cached version, so technically they'd be "within their rights" to keep using it... as long as they aren't getting it from your server. Something you could try: Change the name of your .js file, disallow it from the very beginning, and delete the old one. Then see if the errors dry up.
If you our on an Apache server and you have the rights to edit your .htaccess file, you can use an X-Robots "noindex" directive. It is a bad practice to disallow Googlebot with robots.txt to access css or js files, as it seems like you are trying to hide something.
<FilesMatch "\.(css|js|txt)$">Header set X-Robots-Tag "noindex,follow,noarchive,nosnippet"</FilesMatch>I use the .txt too, for the case someone links to my robots.txt file.
Nice. I'm gonna try to implement with the ISAPI_URL Rewrite. to disallow robots for those files...
Is there any benefit in putting nofollow's on links to the same page?
For example, I could have...
Product image link
Product name link
"Click here" link
...all going to the same page.
Is it worth no following the 'image' and 'click here' links to reduce my links per page but keeping the rich anchor text 'product name' link?
Some people said in google groups that the nofollow change affects only the internal links and not the external (that you do not loose pagerank for nofollowing external links). What is you opinion on this?
If that is true or not, it is a very healthy practice.
Sorry, guys. I meant to click on Add New Comment and hit Reply instead.
Thoughts on a link to an image with a corresponding text link - nofollow or what?
Personally, I would NOT have been nofollowing those image links... even before the recent news. I recommend removing the nofollow from both links... and then implementing one of the following: 1. Two links. Use the anchor text from the text link as the alt attribute of the image link. 2. One link. Put the image tag AND the text inside the anchor tag. Option 1 is better, in my opinion. Option 2 is more for situations where a CMS limits your ability to use custom alt attributes.
Thanks. So, you are suggesting just having two links, one an image and one text, both pointing to the same URL? I would think that you would want to be sure the text link got the credit. Some folks seem to think that the first link the spiders find is counted and the second link to the same URL is discarded. In this example, the image link would be crawled first.
Also, would that mean that if you had 50 images on the page with 50 corresponding text links that the link juice is divided by 100 or are the duplicate links ignored?
I appreciate the feedback.
The 2-links-per-product setup is fairly common with ecommerce websites, and I would expect Google to recognize that by now. So my recommendation of 2 links is based partly on that assumption. Additionally, I've done a few informal experiments regarding the weight of text links vs. image links, and I haven't seen any difference as a result. But obviously, the image has to have the alt attribute in place. For example, I've tried changing this: <a href="URL">KEYWORD</a> to this: <a href="URL"><img src=".JPG" alt="KEYWORD" /></a> for links across an entire site, and nothing happened. (Plus, when you look at the cached text version of a page, you'll see that Google has changed all the alt attributes to regular text links. That's not proof... but it's a good sign.) If you only want to use one link per product, I'd go with Option 2 from my previous comment. Regarding your questions about which links get counted (1st, 2nd, or both), I don't think the SEO industry ever came to a solid conclusion on that. Your best bet at this point is to: 1. Avoid doing anything fancy that Google might not recognize. In other words, stick with what most sites are doing. 2. Avoid using the nofollow attribute on internal links. EDIT: deleted extra </a> tag in code example.
Since I first heard of PageRank sculpting via no-follow I've had a bad feeling about it, and I'm glad they've foiled it.
It has always seemed a little grey hat to take something that Google made so that you can essentially say, "Even though I'm publishing a link to this site, I'm not voting for the quality of the content." and using it to say, "this is how I want your algorithm, which is designed to try to improve the user experience and pull relevance from actual content, to interpret my website's relevance.
And I also think that the idea of No-Follow and how it was thought to (or did once) interact with pagerank was making SEO minded web professionals to be greedy with their link juice... which will kill the system of sending votes to others websites.
Now that they have ensured that no-following links will not increase juice saturation in the other links, webmasters can use no-follow in the way it was intended... to prevent voting with your pagerank to sites you don't approve of.
~ Joseph Sims
As a new SEO, I am just wondering what happens to the page rank left over from nofollowed links in this case?
Update: Sorry for dublicated question, I guess this is the same issue written justover my comment :(
I've got a client that has a large number of nofollowed internal links in their footer navigation, amounting to about 25% of links on the page.
I've aked them to remove them.
If what Matt Cutt's is saying is correct I would expect more PR to be flowing around the site as it will no longer be flowing down a black hole.
Hopefully we will see a corressponding increase in ranking and search visits.
As the site is updated daily Google crawls the site frequently and they get pretty good taffic (over 500K organic search per month) so noise should be kept to a minimum.
Should have some results to report shortly, assuming their developers sort it our quickly...
That was the first thing that came to my mind. Having footer navigation in addition to your primary nav seems even less appealing now.
This is definitely the most important post I've read all week/month. Thanks for the great coverage as always.
thumbs down. i can't believe you're recommending an iframe for anything.
There are plenty of valid reasons to use iframes. Do you also disapprove of AJAX, Web 2.0 applications, and quick page load times?
My web site doctype is XHTML+RDFa. How can I use Iframes there?
1. I'm calling bullshit. 2. If your site really does use RDFa, you can use iframes the same way everyone else does. W3C's RDFa primer says:To date, because XHTML is extensible while HTML is not, RDFa has only been specified for XHTML 1.1. Web publishers are welcome to use RDFa markup inside HTML4: the design of RDFa anticipates this use case, and most RDFa parsers will recognize RDFa attributes in any version of HTML. The authors know of no deployed Web browser that will fail to present an HTML document as intended after adding RDFa markup to the document. However, publishers should be aware that RDFa will not validate in HTML4 at this time. RDFa attributes validate in XHTML, using the XHTML1.1+RDFa DTD.
You thumb down my post just because I just asked a question?
What are you calling BS? That I have setup a test page adding an iframe as you have advised me (even if I knew it was wrong) and the markup of the page did not validate?
Instead of playing expert in every field, please show me a life example that it works. When I say it works, that it validates.
That said, you better practice web standards instead of posting misleading, inaccurate and incorrect information, misinterpretating quotations or W3C.
Thanks.
Hey guys - just a reminder that we love spirited debates and good back and forths, but it needs to remain professional and not get personal. Darren - no need to "call bullshit" - there are far friendlier ways to disagree and we'd love if you could switch to some of those. :-)
We want to make SEOmoz as welcoming a place as possible - if you're seeking more of a conflict-approving zone, there are lots of others places on the web that permit this.
Thanks for understanding!
1. Okay, I retract my "bullshit" statement. What I meant was... I find it very hard to believe that anyone has a website that: a. Uses the XHTML+RDFa 1.0 DTD, and b. Validates, and c. Needs PageRank optimization, and d. Requires the use of iframes. I assumed you were just trying to be a wise guy. Honestly, I still don't believe such a site exists, but the point is... I should have expressed my skepticism more subtly. 2. I don't think I posted any "misleading, inaccurate and incorrect information" in my previous comment, especially since your question made no mention of W3C validation. Also, I made no attempt to interpret the quote from W3C--I simply copied and pasted it, allowing for readers to interpret it for themselves. In fact, I referenced the W3C specifically because I know I'm NOT an expert in document type definitions... nor do I claim to be. With that said, I'll give your question another shot. XHTML 1.1 may not include (i.e. validate with) iframe elements by default, but you can extend its definition or even define your own DTD. In other words, the "extensible" characteristics of XML/XHTML give document authors the power to control what is or isn't considered "valid." But again, I'm not an expert, so I encourage you to consult the following resources for more information: Modularization of XHTML Developing DTDs with defined and extended modules
First I want to make clear here that I do not feel confortable with your offensive aditute. If this will continue, I will cancel my membership here and never come back again. OK?
Am I trying to be a smart guy? No need to do so.
Such a site does not exist? WRONG! My site is XTML+RDFa. And validates. Check my web site url in my profile.
I am still working on the site adding triples, microformats, etc. It is a huge work, but in long terms it makes a lot of sense.
And I do not need Iframes.
Are we clear now?
Seoworkers, I don't understand that response. Maybe you misread my tone? My previous comment was borderline apologetic, yet it seems to have angered you even further. So let me make myself clear: I'm not trying to offend you. I am also an advocate of web standards, semantic web, and accessibility... and I definitely understand the extra effort it requires, so kudos to you for setting the example with www.seoworkers.com. The only thing that's still unclear is why you're asking about iframes if you don't need them?
My apologies if I have misunderstood you. I am glad we could get this out of the way.And thanks for the kind words about my web site.
Now about the iframes. I am also looking for solutions to block bots to follow certain links like affiliate or paid links and that would have the same effect, and that it is not violating any web standards and/or serach engines guidelines, and that they are also are still accessible and usable for users.
A long time ago I developed a sort of alternative to the nofollow attribute for the purpose above, but I am not happy with that. To be specific I am not happy with the url accessibility and usability.
I posted that recently at WPW forums, where you can have a look: https://www.webproworld.com/search-engine-optimization-forum/78553-new-canonical-tag-big-3-a-4.html#post440204
I know it is not the alternative to discount a link entirely and that there is a leak of PR juice so far I can tell. Or? But I am sure you know what I am about.
Can you have a look and tell me what you think?
Thanks buddy. :)
With all of the sculpting going on nowdays, I'm actually really surprised that we didn't see a major flux in rankings "more than a year" ago when they made the change in their algorithm. There are so many implications here:
Why didn't webmasters notice increased traffic to those internal pages that were "hidden" previously with nofollow?? My Take: this will discourage webmasters even more from linking out to valuable resources and will actually create more spam in the long run.
"All of the millions of published spam comments now get PR passed... (that is madness!)"
Really? Doesnt Cutts say this:
“[*] Nofollow links definitely don’t pass PageRank. Over the years, I’ve seen a few corner cases where a nofollow link did pass anchortext, normally due to bugs in indexing that we then fixed. The essential thing you need to know is that nofollow links don’t help sites rank higher in Google’s search results.”
So why would that be?
yeah... just re-read his post. Trying to understand all this
Do you think this has become a purely negative action - you can only remove value (nofollow = no juice), but not add any value by doing so?
Ugh this sounds like big mess. While you're doing your test, Rand, you should experiment with linking to external sites.
It would be sweet to be able to create an index of external sites, ordered by their ability to bring you more traffic. It's a long-shot, I know.
Rand, this is a really helpful post. Looking forward to your test results. Basically it means that I have to start testing on my own again rather than relying on what official best practices are. What a pity, really. With Nofollow, we were actually going somewhere...
hi rand,
I read various blog post and all the comments posted on here as well as matt's blog. This is very disappointing especially for SEO's working on advance techniques as Google are changing algorithms without backward compatibility.
The solutions provided by you to keep blocking links getting indexed are good and this is the only alternative solution of link sculpting.
As far as i know, i will keep doing natural and thematic link building and i am sure it will rock.
I think this move will force webmasters to do a much more controlled version of sculpting on page, reducing the number of links per page.
so no we have a per page issues with link, we also know footer and templated navigation links are devalued at different degree's
We' see temporary spike in ajax, flash, iframe links espcailly from UGC, there also is some strong potential for some black hat style cloaking here.... anyone working on a wordpress comment Iframe plugin yet? (you heard it on SEOmoz first!)
I also think we'll increase the number of links in content that are really not contextual realvent just to get the links to other pagesand not loose the value of footer or navigation.
Lastly I believe the big silver bird will return if I could just figure out what to do
one other point, in matts post, he says he uses 'nofollow' only for his RSS feed, he does use rel="external nofollow" for every link on the page.
I can't find an RFC on nofollow but is google now treating rel="external nofollow" different then rel="nofollow"?
Presumably something like Disqus would work to give that effect as the comments are off-site and included via javascript (AJAX call I think). From the Disqus site
So it seems that if they turn off the indexing on Disqus then all comments are cloaked.
I have 2 questions:
Q1. So what?
If you have a page with 60 PR points and 4 out-links. Those link would get 15 points each, right? Now nofollowing 2 of them would leave 58 points to divide... At least that's what Matt said. My point is: this also needs to be investigated. If the estemed colleagues didn't see a change last months, then this could be a reason. My thought on this: the nofollowed links don't evaporate all of the juice they would otherwise get.
Q2. So an "outgoing link" is also an external link? In one of his answers to questions Matt suggests, that external links are treated differently. A lot needs to be tested :)
Danny Sullivan 's comment on MattCutts blog really stands out, he says that : “Google itself solely decides how much PageRank will flow to each and every link on a particular page. The number of links doesn’t matter. Google might decide some links don’t deserve credit and give them no PageRank. The use of nofollow doesn’t “conserve” PageRank for other links; it simply prevents those links from getting any PageRank that Google otherwise might have given them.” I'm in no way on the level of those guys but it seems a very solid view to hold onto.
Yeah, I read it after I commented.
What's more: Matt Cutts confirms.
Danny always stands out.
I'll preface this by saying that I'm still sorting out the details like everyone else, but my current understanding is that the PR would still be split 4 ways, but that the nofollow'ed link juice would evaporate. So, if you've got 60 "points" going to 4 links (15 each) and nofollow 2 of them, 30 point evaporate and 30 are left for the 2 followed links.
But how could it be, that when not all links weigh equal (confirmed), these nofollowed links would evaporate the same as all others? My guess is, that the algo is much more complicated than that. And that the rankpoint system Matt mentioned isn't a simple proportional thing. And he simplified the example for better understanding of what is going on.
Oh, absolutely - this is a gross oversimplification. The gist is that the nofollow'ed links are now being "counted" - instead of having their link juice flow into followed links, they simply lose that link juice into the void now.
And what is left for the live links?That is my point. You lose some, no question, but how much do you win if done right? Nofollows in the right places, dofollows on other (to links to the same pages). Endless variations are possible. One more successful than others.Again, you lose some, but the win might just be enough to do it anyway.
This is my understanding / perception too, leafing 30 points, not 58.
I would also point out that at SMX, when Matt was asked "then what happens to the leftover PageRank?" his answer included the word "evaporates"! That seemed pretty harsh to me...
So this is just about seo's looking dumb to their clients now that they have to recommend something else, right?
None of their options sound very 508/accessibility friendly (or at least easy). "An embed in Flash, Java or some other non-parseable plug-in"... I mean, come on. Non-Parseable? Maybe the iframe solution. But yeah you block that in robots.txt and a gazillion other bots/scrapers what have you that don't respect robots still picks it up. 10 bucks says Google revisits their stance on this sooner than later.
Thanks - good subject.
~ Jim @SEO_Web_Design
Why would Option B work, does Google only count links to valid and accessible pages? Therefor they'd need to check all pages that a page links to.
Rand, thanks for the thorough explanation of this issue and possible effects/outcomes. It will be interesting to see the results of this change across the web.
So here's my question:
My site currently has three links to the homepage on every page
Does this help steer the flow of juice back to my homepage moreso than a single link to "Home"?
I could easily add at least two other links back to my homepage without detracting from the user experience:
Let's see who's gonna be the first to release a flash based commentary plugin for WordPress, or maybe there is already one around?
After thinking this over for most of the afternoon I have to wonder whether this whole issue is being blown slightly out of proportion.
Given that the change was announced over a year ago and no one really seemed to notice, in our agency it hasn't had an effect on best practice and for the most part the only people who seem to be confused are those not familiar with the practice of PR sculpting already, it would appear that a combination of over analysis and scaremongering may have led to a perhaps slightly sensationalist take on the annoucenment.
I'm not directing this at SEOmoz or anyone in particular as there are many posts about it across the SEO community, but much like the so called 'revolutionary' canonical tag this appears to be (until significant results prove otherwise) a flash in the pan.
Very well written article indeed, thank you so much for sharing such information with us, i hope we will see more from author in the future. Cheers.
Wow I have never heard of this before, what an interesting info.
As a sometimes conspiracy theorist I have to associate 1) the delayed release of this information and 2) the direct relationship that Google places between nofollow and SEO's with this post from Outspoken Media. https://outspokenmedia.com/seo/google-profiles-seo-as-criminals/
It really is a shame that all the potential positives that nofollow brought into the SEO world are being tossed out the window with this pseudo-new announcement.
Also, I would like to openly call for a new tag since nofollow isn't going to work like we want it to. There is NO reason why we shouldn't be able to implement a rel="nocount" rule that eliminates a link from the denominator.
No reason why we can't implement rel="nocount" ?
Well, I suppose we could implement it - what we can't do is force Google to take a blind bit of notice. Otherwise, we could get them to implement rel='count-double'
tim, there is plenty of code that replicates what you want with nofollow.
@steve No Doubt, didn't say there wasn't.
@alun I see benefits from Googles perspective on having people use nofollow. When sites use something else to accomplish the same end Google has less information about the nature of the web to use in its algorithm. There's a tradeoff to be had between Google collecting my nofollow related information and allowing intended flow of Page Rank with a rel tag. If Google see's a loss of valuable data as a result of sites linking to each other with Flash, Javascript and iFrame, would that be enough incentive for them to implement a similar tag?
Oh, and don't get me wrong, I'm all for the count-double, but I think that might be a little more difficult to sell them on.
i was wondering, isn't there an addon to most server software, i.e. apache, iis, that will put up a page inbetween links, i.e.
WARNING.. you are leaving our website..
and if you nofollowed and no indexed that page inbetween, wouldn't it do the same thing?
If you tell Google not to index the page in between links and not to follow i.e with robots meta tag directives like noindex and nofollow you will have a problem. Google will not index the page, but don't forget that the noindex directive acrues PR. And if Google cannot follow the link or links of that page, you create an as known dangling page or node, with the result to leak PR.
I developed a solution which I mentioned above and posted at WPW forums here https://www.webproworld.com/search-engine-optimization-forum/78553-new-canonical-tag-big-3-a-4.html#post440204
For my understanding I cannot see how PR leak could be possible with it.
Any thoughts?
Matt Cutts June 15, 2009 at 10:55 pm "Ben Finklea, this is a change that’s been live for well over a year; if you’ve got a site that works for you and you’re happy with, I wouldn’t worry about going back to change a lot of work." I do SEO for a website with several million indexable pages, indexation issues are my bread and butter, we have seen very positive results those past months using a combination of changes including nofollowing internal links, this means that either nofollowing internal links never helped indexation or that if it helped, site architecture and general crawlability was largely the critical factor. Anyway we are redisigning and i still believe PR matters in some way for indexation so we will use old school javascript redirects or iframes as Rand points out.
I'm finding it very hard to imagine several million pages of quality content on a single site. Forum site? Photo site?
I'm guessing Google have noticed that sites with a high links per word per page have lower quality (higher bounce rate). I'm wondering now about stitching a page together in the style of 1997 frames but using iframes ... hmmm.
I know many sites that are largely duplicate content and scraped links that have a much lower bounce rate than most blogs.
Nice info...
Need learn a lot about this SEO thing... :(
Thanks rand!
So that is going to mean the Siloing is coming back? I have always worked on the idea that linking out to good places not only sets you as an authority in your industry but passes good karma
RE: Siloing is coming back? [Ha ha ha ha ha ha ha!] No. This definitely does NOT mean siloing is "coming back." Siloing (in theory) used the nofollow attribute to channel topical relevance and create "themes" within a given website. The so-called practice of "themeing" a site died when Google gave the following answer in October, 2008: Q: Let's say my website is about my favorite hobbies: biking and camping. Should I keep my internal linking architecture "themed" and not cross-link between the two?A: We haven't found a case where a webmaster would benefit by intentionally "theming" their link architecture for search engines. And, keep-in-mind, if a visitor to one part of your site can't easily reach other parts of your site, that may be a problem for search engines as well. The only way siloing might have improved rankings is if it was inadvertently channeling PageRank to landing pages. But in light of Matt's announcement, it should now be clear that siloing is dead. Or in other words: siloing = creating a hierarchical information architecture that gathers small chunks of internal PageRank and sets them on fire.
I believe you are misunderstanding the original concept of what it means to use siloing on a site.
Siloing is simply doing what Matt recently advised when asked if we should pagerank sculpt - spend your time picking the best links for your main pages, to send your pagerank there.
Siloing is nothing more than providing graduated levels of internal linking that are contextual to the section of the site the surfer is in. You create a silo with links, and if you were trying to create them by nofollowing other links, it was never a good approach, and could never be comprehensive.
Link laterally to similarly themed pages, ones that share a large stem of the breadcrumb navigation - do this more than linking to pages further away in theme (or silo), and you eventually create the silo shape with your links.
You say that siloing might have improved rankings if it inadvertently channeled pagerank somewhere - but there is nothing inadvertent about traditional silo linking. You're linking up to partent category pages from lower level pages, you're channeling your pagerank there without subtlety. You're linking sideways to create relevance, and upwards to push PR - then you simply optimize the higher level pages for more competitive keywords, and if you pushed your pagerank (and other quality indicators) at them properly, you'll have a better chance of ranking on competitive stuff with your hub pages.
That's why we silo, to push juice up, to enchance relevancy and popularity of pages that need more to compete. It has very little directly to do with the nofollow attribute.
Damn, this is well said. You should see the looks I get from clients and other seo collegues when I say to do this! It is natural and commons sense for your syndication strategy for your custom, dynamic, content. This is why I also promote big websites!
OK so in short adding the nofollow to a link will stop Google from passing juice through that link but that nofollow link can still greatly change the weight and ranking abilities of the page it resides on.
RIght?
Don't think so, if I understand what you are saying.
It changes the way the other links influence the rankingabilities of pages they link to.
Rand says:
but I've a client who originally had a pure Flash site (agh!) and Google seem to index that fine (though they only use the terms to locate the wrapper page). I'm sure they read Flash links.
Yeah - it depends on how you build the Flash files. It can certainly be done to prevent search engine access.
Thank you Google for making my job as an SEO even more important than ever before. You rock!
This is more in line with the original intent of the NoFollow tag - it is intended to prevent PageRank flow to pages for which a webmaster cannot vouch. It was never intended to be used to discredit your own pages.
I'll be watching for the results of your tests. Sometimes the results don't match.
Thanks!
In summary... Screw page rank sculpting and screw the no-follow microformat. As I said, when they first came out, they are both a total load of B*$£$"X!!!!! No body should be using no-follow or page rank sculpting, if you can't do it the proper way then your wasting your time.
Reading your comment was a waste of my time.
Very glad to see clarification on this issue.
Looks like I've got my work cut out for me over the next few weeks accessing the use off nofollow on my client sites...
google is changing itself to better understand the web. Which is what I like about google.
Cartuchos de Impresora
- Casey Removed Link
And what I hate is one line blog posts including link dropping.
Especially when the one-line post is something so stupid and meaningless.