The SEMNE organization landed an incredible opportunity last week with a guest lecture from Dan Crow. I've seen Dan speak once before several years ago and was shocked by his overly secretive conduct. Thankfully, Google, and apparently, Dan, have changed a lot since then. He had some remarkable things to share with the SEMNE group and, thanks to Jill Whalen's post (Getting Into Google), the rest of us as well:
We were also treated to some additional tips that many people may not have known about. For instance, did you know that you could stop Google from showing any snippet of your page in the search engine results by using a “nosnippet” tag? And you can also stop Google from showing a cached version of your page via the “noarchive” tag. Dan doesn’t recommend these for most pages since snippets are extremely helpful to visitors, as is showing the cache. However, Google understands that there are certain circumstances where you may want to turn those off.
Google is coming out with a new tag called “unavailable_after” which will allow people to tell Google when a particular page will no longer be available for crawling. For instance, if you have a special offer on your site that expires on a particular date, you might want to use the unavailable_after tag to let Google know when to stop indexing it. Or perhaps you write articles that are free for a particular amount of time, but then get moved to a paid-subscription area of your site.
Lastly, Dan mentioned that Google still isn’t doing a great job of indexing content that is contained within Flash and/or AJAX. He said that you should definitely limit your use of these technologies for content that you want indexed. He provided a bit of information regarding Scalable Inman Flash Replacement (sIFR), and explained that when used in the manner for which it was intended, it’s a perfectly acceptable solution for Google.
As an aside, I'd like to point out that Dan's approval of the sIFR mechanism, which replaces HTML text (shown to search engines in the code) with Flash text (more visually customizable), strongly suggests that CSS image replacement is also an acceptable practice. However, I believe that Google's position on both methods for customizing visuals of text on a website would demand that the readable text in both cases exactly match - this isn't a good way to create hidden text, just a good way to improve user experience. Jill's post continues:
Dan provided us with some insights as to what the supplemental results were and how you could get your URLs out of them. He explained that basically the supplemental index is where they put pages that have low PageRank (the real kind) or ones that don’t change very often. These pages generally don’t show up in the search results unless there are not enough relevant pages in the main results to show. He had some good news to report: Google is starting to crawl the supplemental index more often, and soon the distinction between the main index and the supplemental index will be blurring. For now, to get your URLs back into the main results, he suggested more incoming links
This is some excellent material from Dan and serves as further evidence of increasing lines of communication between webmasters and Google. I'm crossing my fingers that Yahoo!, MSN & Ask make similar strides in the near future. And, clearly, Seattle needs its own professional search marketing organization.
p.s. Jill's requested that the following accompany any quoting of the post:
CEO and founder of High Rankings®, Jill Whalen has been performing search engine optimization since 1995 and is the host of the free High Rankings Advisor search engine marketing newsletter, author of “The Nitty-gritty of Writing for the Search Engines" and founder/administrator of the popular High Rankings Search Engine Optimization Forum. In 2006, Jill co-founded SEMNE, a localsearch engine marketing networking organizationfor people and companies in New England.
There is also the interview that Dan gave to The Guardian back in March. He talks about flash and crawling. Good reading.
https://blogs.guardian.co.uk/technology/archives/2007/03/27/interview_googles_dan_crow.html
Great interview. I think it's pretty cool that in March Dan was saying that people were requesting "unavailable after" functionality, and now we have it.
Nice to see Google playing well with both sides of the search marketing industry.
I also had the good luck to attend the SEMNE session - the degree of openness was actually quite shocking.
Dan's views on Flash patching were not quite as rosy as his sIFR comment would suggest. He characterized SWFobject (a cousin of sIFR) as "dangerous" - and actually mentioned cloaking in relation to it.
I've posted some notes at out blog, Search Matters.
Would be interesting getting more specifics on this, though my immediate assumption is that it may be less in the "use of" than in the "implementation"... I think the key issues comes down to whether whatever degradation model is used is a true and comparable representation of the Flash, or it is used to feed the bots.
Prior to Dan's talk, I'd waivered back and forth on SWFobject myself. My latest take was similar to yours: if the two versions are sufficiently identical, then there shouldn't be an issue. But after hearing Dan's stance, I have to concede that there's no way for the algo to be sure that the two version are both identical and legit.
For example, I can dump a lot of keyword text in the HTML version, and also dump the same text into one of my Flash panels - a panel that would only be seen if you happen to click a single pixel in the animation, at just the right moment.
I'm guessing that they see too much room for trickery here, so they're trying to discourage its use.
If we going to use PigeonRank ™ we might as well do it over TiSP
https://www.google.com/tisp/install.html
Is anyone else surprised by the stale content supplemental reference?
Could it be a combination of these two factors rather than an either/or? Plenty of old stale pages manage to remain in primary index.
I'm with you there - you need to be both stagnant and unimportant to stay "supplemental." We've seen what appears to be evidence that even without a lot of link juice, frequent changes/updates can keep a page in the main index.... However, that said, it doesn't neccessarily mean its ranking well.
RedCardinal,I’m not surprised, but it sounds like a problem / issue to me. For example if you were researching the first Gulf war, then the most relevant content might be many years old first hand reports. If they are deemed less relevant because they haven’t changed in years, then what do you get instead? Wikipedia articles written by lifeless dilettantes. Just because information is old doesn’t always indicate that it isn’t the best
That statement by Dan opens up a whole can of worms. There are probably other factors Google isn't disclosing.
For example, Tedster over on WMW mentioned a patent named "The April 2007 patent for Document Scoring Based On Traffic" and wonders:
My first inclination was to dismiss his idea but after Dan's statement about page freshness also being a factor anything is possible.
That could be read another way: the links don't change much - or the amount of links to the page stays the same without new ones coming in. If a page's relevance drops off the link count will too.
I'm not sure I'm explaining this well. And even if I am I'm probably wrong.
That statement made me do a double-take as well.
Lots of great stuff here, but that statement was especially disturbing. I suddenly have this vision of webmasters everywhere making constant but relatively meaningless content changes to pages for the sole purpose of trying to influence the engines.
One step forward, two steps back for SEO?
I know sIFR is old but i've only just discovered it!
My question is why do people not use this?
If sIFR is both accessible and seo friendly why is it still so uncommon?
Thank you for an interesting post!Perhaps you or somebody else please could tell me how to get a similar listing like displayed on this link? https://www.google.no/search?hl=no&q=joomla+seo&btnG=Google-s%C3%B8k&meta=
As you see the website JoomlaSEO.net website has a list of related websites linked as "sub items" inside the Google listing.
I'm very interested to know how this is done.
I'm also interested in posts regarding image and video listings inside the regular listings.
I'm available at mail ( @ ) mesterweb (dot) no
Or by using my contact form on this website
Best regards
Trond L.
"p.s. Jill's requested that the following accompany any quoting of the post:
... And then an opening inverted comma? What am I missing there, or is something not displaying for me?
Thanks. These are great news. Especially these about the supplemental result spyder visiting.
And here I thought that all you had to do was use duplicate content to stop Google from showing any snippet of your page :)
But all jokes aside - Jill's post is a great read... Thanks Rand for bringing this to our attention.
Great article. The snippet and unavailable tag are both news to me. Like eCopt said I will store that information away until I need it.
Unless you are building something like google maps them, If AJAX and javascript are applied to a site correctly then there are should br no issues with being crawled.
Everything on a site should work without having javascript switched on and then it sits in its own layer on top adding a functionality layer.
Accessibility is a big issue where I work :)
Thanks for sharing Jill's post Rand. That is cool. Never knew about the nosnippet tag till now (probably won't use it too much, but still good info to have stored away). Recently read about the unavailable_after tag on Danny's site, looks interesting and much needed for seasonal pages, promotional events that expire, etc.
Still no falsh and Ajax, huh? Oh well, could be worse. Heard they are making progress on being able to 'see' valid Javascript. Heard about the news on supplementals on Matt's site and some random commentary. I am glad they are either doing away with it or crawling those pages more often, although I see why they have them, it's not really proving to be very much in need given the size of their index now.
Dan’s recommendation to throw more links at your Website violates the Google Quality Guidelines, which states one should not try to manipulate Google Page Rank algorithm.
Dan, needs to read what his team members write in Google Webmaster Guidelines maybe then he will sound authoritative and not idiotic!
Why people have a hard time understanding what you need to do? Google tells you to build original content pages, avoid duplication, and be authoritative on the subject.
Supplementary = Duplicate = Spam
I feel like I am beginning to Spam Google by saying this, but we need to drill it into the Talking Heads!
I get people linking to my pages because they like the content, they like my work as a developer, they like my company’s service.
People add my Websites do directories, proxies link to our Websites.
IMAO I get links and links up the cazoo!
But having all these links to a Website does not make it more authoritative and does not bring more search engine hits.
What does is content, content, and content.
If humans find a Website useful so will the search engines.
Hey, what comes first? The chicken or the egg?
Time for a cheese toast, its 7:00 AM in Japan now.
Quite right, Igor: links don't matter at all. We all know it's about the pigeons.