/rant on
Ok. That's it. I've had enough. When, exactly, did webmasters lose control of their content and why is it that the search engines now get to define how we have to provide that content?
I understand that the search engines would like to provide relevant content to users.
But this holy war against paid links and any other issues that the 'experts' at the search engines don't like is getting a bit ridiculous isn't it?
If you're wonder if I've suddenly gone mad, here's what I'm talking about.
There are a few interesting things in this article.
The first of course is this quote:
...Rae Hoffman illustrated that other SEOs could easily see paid links using open tools like Yahoo’s Site Explorer, so it doesn’t even require the special tools that a search engine has.
Of course, quite a few others have discussed the paid links bit ad infinitum and it's basically come down to most folks disagreeing with Google on this issue. Not so susprisingly, Matt continues to take every chance to pound his pulpit on this one :)
Then I saw the discussion on subscription based content:
Subscription-based content
Googlebot and other search engine bots can only crawl the free portions that non-subscribed users can access. So, make sure that the free section includes meaty content that offers value. If the article is about African elephants and only one paragraph is available in the free section, make sure that paragraph is about African elephants, and not an introductory section that talks about sweeping plains and brilliant sunsets. If it’s the latter, the article is likely to be returned in results for, oh say, [sweeping plains] and [brilliant sunsets] rather than [African elephants].
And compare your free section to the information offered by other sites that are ranking more highly for the keywords you care about. If your one free paragraph doesn’t compare to the content they provide, it only makes sense that those sites could be seen as more useful and relevant.
You could make the entire article available for free to users who access it from external links and then require login for any additional articles. That would enable search engine crawlers to crawl the entire article, which would help users find it more easily. And visitors to your site could read the entire article, see first-hand how useful your articles are, which would make a subscription to your site more compelling.
You could also structure your site in such a way that the value available to subscribers was easily apparent. The free content could provide several useful paragraphs on African elephants. Then, rather than one link that says something like “subscribe to read more”, you could list several links to the specific subcategories availabe in the article, as well as links to all related articles. You could provide some free and some behind a subscription (and make the distinction between the two obvious).
Now, it's really rather subtle and perhaps I'm nitpicking but it seems to me that we're being forced to provide our content in a particular way (once again) so that the Search Engines can more easily deliver it.
And that, my dear reader, is wrong six ways from Sunday! It just really annoys the hell outta me when the Search Engines attempt to define how we must behave or we're considered the bastard stepchildren.
/rant off
Hmmm...maybe I need some Kava Kava?
- G-Man
Matt,
This topic does concern me quite a bit (though not as much as the continued ranking of supposed authority sites like stanford.edu using student sections as doorway pages). Anyhow, the key problem I have with your argument is that Britannica should modify the content deliverance methodology in order to achieve the desired results, rather than Google modifying the binary to intuitively understand that "hey, britannica knows more about this stuff than wiki, even though we can't see all the text" -- I understand that we can throw in discussions of linkage, use of various design elements, etc, but I have to agree with Geoff here: a search engine's job is to seek out and organize according to relevance; a websites' job is simply to be relevant (not algorithmically relevant).
Cygnus
>The key problem I have with your argument is that Britannica should modify the content deliverance methodology in order to achieve the desired results, rather than Google modifying the binary to intuitively understand that "hey, britannica knows more about this stuff than wiki, even though we can't see all the text"
Well I don't think it is just about who knows more...I think it is about who is best able to answer the query for the most people. Most people don't want to search and wind up on subscription pages if good enough information is available somewhere else free. And most people using general search are likely just looking for good enough information, instead of the exact perfect best authoritative text on the topic.
Most people are too cheap to buy better stuff, some even valuing their time at next to nothing to save a few dollars on rebates or coupons and the like. Most people are too lazy or time limited to read books.
If Britanica wants Google's search results to lead searchers to subscribe pages I recommend they increase their AdWords budget.
I believe GeoffreyF67's blog entry on cloaking's benefits (https://www.seomoz.org/blogdetail.php?ID=1014)...are particularly valuable in this case. Feeding Google an index,nocache and then cloaking all other traffic would allow a best-of-both worlds scenario. Google could identify the value of the content behind-the-scenes to accurately compare it with Wikipedia, et al. without violating the subscription model.
While I sympathize with the argument that Google wants to provide accessible results to its users, it is a fundamental flaw in the overall user-experience if Google essentially excludes the entirity of pay-for-information models which have existed since the first book was ever published.
This issue has blown up internationally on the issue of Google News (Agence France Presse). Google just needs to be more conscientious of publishers who make a living by providing information.
Dashes, underscores and dots? Not sure about all this, I have had everything on my site since it began, and has never seemed to make much of a difference. The changes which have had the biggest impact that I can tell have happened in the last 6 months ... Google changed its algorhythm again and my site was sent to the abysss overnight. I questioned Google about this and within 2 days it had returned.
Now while my site www.airscene.co.uk does not have a lot of competition (airshow dates) it has always manager to maintain a top 10-20 position in Google and all the other major search engines. Apart from the time I mentioned this has never once been affected. It has a low page rank by all accounts and I dont provide oodles of text content to increase this.
HOWEVER in recent months Google seems to have stopped crawling the whole site, now it is only the index page. What is more, when I enter site:www.airscene.co.uk I google now show pages from 2003??? These pages have not been returned since mid way through 2004, so why should they now show up again??
On another note I have been helping a friend build a new site over the last year or so, and find that both MSN and Yahoo crawl and index much more of the site than Google do, and (I imagine this has been questioned many times) we suspect that Google is protecting the interests of its Adwords customers... do a search for anything from hotels to solicitors and see how far down the rankings a non-adword site comes. Several pages down in more than a few instances. And new sites offering the same content do not have a chance even if they are optimised and follow all of the Google golden rules.
I for one look forward to the demise of Google, and have removed it from my favourites list, I am fed up hearing from Google how they provide the most up to date content, when they point blankly refuse to crawl sites which arent paying them.
G-man, I think you missed the thrust of Vanessa's notes on this one. Britannica only shows the first 4-5 sentences from each article, because they've chosen a paid business model. The whole panel (two SE reps and two SEOs) were offering suggestions for other ways that Britannica could help make their content more visible in search engines while still preserving the model that Britannica wanted to follow. Vanessa was just documenting many of the suggestions we gave.
For example, we recommended that Britannica find the most relevant paragraph (either by keywords that showed up in server logs, or by information content, or by capitalized phrases, or by statistical improbably phrases such as Amazon does) and show that instead or in addition to the first paragraph.
So I think it was completely the other way around. Rather than trying to force a site owner to provide content in a certain way, Britannica had already decided their business model (paid sign-ups), and everyone on the panel was just trying to give them advice that would preserve that model but increase their exposure in search engines. We riffed for a long time with various ideas to help (e.g. find the 1% most accessed articles and make those free as teasers to entice people into signing up).
Sorry if it was unclear or the write-up from Vanessa made it sound like search engines were trying to dictate how to do things. In reality, it was a really constructive session, and the folks from Encyclopedia Britannica came up to thank me afterwards, so I think they appreciated the suggestions.
It's an interesting conundrum, how does some who charges for access to content provide enough content to garner links when they "aren't giving away enough" to get links? I think I'm pretty safe in assuming Britanica which was used in the elephant example has better content than wikipedia. So Google's way of doing things has affected the way we percieve, recognize and reward value. I would suggest they cloak ... er IP deliver content but we were recently admonished that was wrong too
Actually, contol over content was lost when the first page was optimized to get higher SERPs. Right?
This stuff doesn't bother me too much because I will work on methods that will succeed regardless of these issues - and my lazy competitors will pack it in. I'll be sure that they see the elephants. It might actually play in my favor. :D
Rand, you might enjoy this stuff too.... when the SEs get tricky more people will read SEOMoz.
Brilliant stuff, G-man, although I have to wonder about intent. My guess is that Vanessa from sitemaps isn't trying to be malicious, but I could be wrong...
Outcome is still the same, though. We tailor content for machines, rather than users.
I really don't think Vanessa is trying to be malicious either (she just happened to get in my crosshairs :) but as you said - the outcome is still the same.
It gets really annoying when I see this type of hogwash shoved down our throats consistently without any type of good old fashioned debate like the net was founded on.