According to a thread at SERoundtable, a few million results in Google (and other search engines) from Craigslist have dropped out due to a change in the robots.txt. The change must be very recent, as I had used Google to search CL just a week before the holidays. It's an interesting strategy from the classifieds giant - If anyone knows the reason, please do tell.
By: Rand Fishkin
Craigslist Blocks Search Engines
Moz News
The author's views are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.
Danny Sullivan has an update
Put this in the "don't believe everything you read" column. Danny's investigation reveals that nothing has changed and the listings are still in the major SEs. The robots.txt isn't actually blocking anything new.
Thanks emad
My guess would be lag between indexing, ranking and someone finally finding a page in the SERP. By that time the classified ads were usually expired, pulled, etc.
Also I believe CL is going to start focusing more on advertising. They have a $125M per year advertising model but only charge %25 of their advertisers a fee. Now what better way of advertising and monetizing your users than by putting more clicks between them and what they are looking for. More impressions = larger ad capacity. Enough users know them to just go directly to the site.
-PK
I'm sure this is a defensive move. With Google (Google Base) and Microsoft (Freemont) poised to enter the classified ad space they are looking for ways to maintain their #1 position. If Google or Microsoft indexed Craigslist and include the listings in their own classified site it will be tough to compete with the money and visibility that the search engines have. Craigslist must be betting that they can hang on to their users without generating new search engine traffic. It's a bold move and one that I think will help them maintain their lead for longer but will ultimately prove to hurt the site. Personally, I found Craigslist through search engines.
Hopefully we will hear it from the horse's mouth directly.
Craigslist is such a powerful site and works so well on its own. Doesn't need to be in the SE's. But lets wait and see what they have to say on the matter.
Dave
Perhaps they feel the same way about Google as they do about Oodle. (edited post with hopefully shorter url)
SiliconBeat
I am very glad to hear this. While not a major problem now, during the Jagger update, craigslist posts started to rise to the top for some fairly competitive key phrases. This will prevent it. If only ebay would do the same thing....