"The spike in searches related to Michael Jackson was so big that Google News initially mistook it for an automated attack. As a result, for about 25 minutes yesterday, when some people searched Google News they saw a "We're sorry" page before finding the articles they were looking for." - Source
First and foremost, let me extend my best wishes to the family and friends of Michael Jackson. I can only imagine the pain of losing a close friend and then having to watch it play out on a global stage. He made an extraordinary impact on the world and although not perfect, he is a teacher even in death (as evidenced by this post).
The following is a timeline of how the news of the Prince of Pop's death traveled across the internet. Not all the times are exact (they might be off by up to 5 minutes) and not every source is included. All times are GMT.
From an internet marketer's perspective, I found this story fascinating to watch unfold. I was impressed by the speed of information distribution and very surprised to see which site posted the news first. Wikipedia is still the fastest news aggregator. It was faster than Twitter and much faster than Google.
A Timeline of How News of Michael Jackson's Death Traveled Across the Internet
19:21 - One of Michael Jackson's employee's calls 911
The next forty-nine minutes are best described as the calm before the storm. The Los Angles Fire Department arrived at Jackson's rented mansion in Bel Air and family members were alerted of the news.
20:10 - (Story Breaks) A small entertainment site called x17online.com breaks the story.
They post photos and a brief story a full 20 minutes before the much larger entertainment site TMZ.com posts the news. Information goes live on the internet. BOOM!
20:30 - TMZ.com posts "Michael Jackson -- Cardiac Arrest"
TMZ.com posts the story on its homepage and the story is distributed to hundreds of thousands of people via RSS. My guess is they paid a pretty penny for the image above and it paid for itself ten fold with all of the links TMZ got from the story.
21:12 - Wikipedia reports Jackson's Cardiac Arrest
A member of Wikipedia adds the news of the Cardiac Arrest to Jackson's Wikipedia article. This is well before any other news or social media source.
21:20 - TMZ.com posts story of death
Report of Jackson's death starts to show up on RSS feeds and eventually Twitter. It is 11 minutes before the first person clicks on a bit.ly link to TMZ.
21:30 - CNNbrk tweets that Jackson goes to hospital
The official CNN account tweets to its 2 million followers that Jackson went to hospital after suffering from a cardiac arrest
21:31 - First bit.ly link to TMZ story
The first bit.ly link about the story is clicked by someone which leads them to the TMZ article.
21:45 - Wikipedia freezes Michael Jackson page
After an explosion of edits to Jackson's Wikipedia article, editors take the step of locking it down in protective status.
21:46 - Wikipedia article discussion has first reports of Jackson's death (Note: Event updated 6/27/09 due to new information)
Wikipedia editors first mention Jackson's death on the article discussion page.
21:50 - bit.ly link reaches high of 2,500 clicks a minute
.png)
Bit.ly link to TMZ hits high of almost 42 clicks a second.
22:03 - TMZ story on Jackson's death is submitted to Digg
A bit late to the game, the story that would eventually go on to be one of the most dugg stories ever is first submitted to the site.
22:11 - TMZ story goes popular on Digg
The story is moved to the front page of Digg where its distribution erupts.
22:19 - "RIP Michael Jackson" tops Trends on Twitter
Story takes the next step and appears on Twitter's Trends. Tens of millions of Twitter users now can see the story.
22:20 - MSNBC.com Confirms Jackson's Death
One hour after the news of Jackson's death hits the internet, the first mainstream news source publishes a confirmation article.
22:25 - CNN.com Confirms Jackson's Death
CNN, out maneuvered by TMZ and MSNBC, confirms Jackson's death.
22:27 - Wikipedia first reports Jackson's death
Wikipedia editors get enough evidence to post Jackson's death.
22:34 - Approximately 2000 mentions a minute of Michael Jackson on Twitter
Mentions of Michael Jackson hit an all time high on Twitter with nearly 1,500 a minute. That's almost 20% of all tweets at that time!
22:38 - Twitter starts to overload. First signs of the fail whale
Twitter starts to falter as a result of the massive spike.
22:40 - First stories of Jackson's death make it on Google News
1 hour and 20 minutes after the story is first posted on TMZ, Google News starts to report the story.
22:46 - Google News Results of Jackson's death start showing up on the results page for the query "Michael Jackson"
Google News results top the Google results page for "Michael Jackson".
22:58 - Googlebot crawls CNN twitter feed
Google starts returning CNN's twitter feed in "Michael Jackson" SERP and provides link to cached version.
23:00 - "Michael Jackson Died" shows up in Google Trends
Google trends updates and show's "Michael Jackson Died" as hottest trending item.
23:18 - 4chan.org goes down
4chan members temporarily overload servers. I mention this mostly because I find it really funny. ;-p
23:47 - "Michael Jackson Heart Attack" and "Michael Jackson Cardiac Arrest" show up as suggested search on Google Homepage for "Michael Jackson"
Indirect news of Jackson's death (if someone types "Michael Jackson") shows up on Google's homepage.
My Take Away:
Google has a really big problem and SEOs need to pay attention.
(Note: I choose Google rather than the other search engines because it leads them in all of the aspects I mention below. Everything I say about Google applies even more to the other search engines. I only have a basic idea of how difficult the technology problems are with the issues below. For better or for worse, I hold Google to a higher standard and I am not afraid to expect more.)
First, a little background information. I believe it was Ben Hendrickson who first mentioned to me the existence of three separate time priorities when indexing the web. He pointed out that the current version of Linkscape crawls and analyzes the slow moving web with a delay of about 4 weeks. (This is damn impressive given an index size of 54+ billion pages.) Blogscape (PRO Only) is much faster and aggregates the fast moving blogosphere of millions of feeds with less than 6 hours of delay. While impressive, we are still trying to catch up with Google and have started to run into the same wall as them. Sites like Twitter, have created a new real-time web. It is only in the order of perhaps hundreds of thousands of pages but indexing it is almost useless with a delay of more than a few seconds.
The events of Thursday demonstrated that Google is falling behind in the emerging real-time web. It was 3 hours and 17 minutes after TMZ first announced Michael Jackson had experienced cardiac arrest before it appeared as a auto completion suggestion on Google's homepage. In the computer age that is a huge amount of time. It is 3 hours and 17 minutes during which consumers may choose to go somewhere other than Google to get the information they want.
As SEOs, we largely rely on the success of Google for our incomes. These are the same incomes that put food on the table for our families. It is easy to think that Google's technology is flawless, after all, it really is incredible. However, it is experiences like the events of Thursday that reveal how truly vulnerable the search engines are.
For me it was humbling,
Teaser: SEOmoz does have a plan for the real-time web and we are excitedly working on it. More information to come in the future. :-)
If you have any other story sources that you think are worth sharing, feel free to post them in the comments. This post is very much a work in progress. As always, feel free to e-mail me or send me a private message if you have any suggestions on how I can make my posts more useful. All of my contact information is available on my profile: Danny Thanks!
I like the timeline and this is overall really good fact reporting . . . but Danny, the conclusion you come to is a bit misleading.
You make it sound like Google is going out of business if it can’t respond to the fast moving web in a sub 3-hour timeframe. And that’s just not the case. That’s not the product they make.
If you want fast moving information then you go to Twitter where you have the eyes of millions of individuals watching for news which *they* feel is newsworthy whereas, in the case of google, the trending and importance is left to crawlers and algorithms. And I don’t know if this is a fair comparison. Because, as we know, Twitter isn’t search.
I was gonna say exactly this.
While I agree that this is true now, it is not the case for the future. Look at the American auto industry or travel agents or even Kodak. While they are not "out of business" in the traditional sense, they did fail to evolve and now are beyond repair.
I am not saying that Google will suddenly disappear, (that is not usually how big corporations fail) I am merely arguing that if they don't solve this problem they are going to be overshadowed by another company that can.
Again, i agree with your basic point, but it is important to realize that companies like Google, Twitter, Facebook and Microsoft are all starting to blend into individual ecosystems rather than individual product pushers.
Google's primary income comes from search ads but it is also putting major money into becoming a phone company (Google Voice), a mobile phone software titan (Android), a library (Google Book Search), an e-commerce middle man (Google checkout). The list goes on.
When your mission it so to "organize the world's information and make it universally accessible and useful" you stop merely being a search company.
I think Twitter, Facebook and Microsoft are all doing the same by spreading their interests.
I think you make a very valid point Danny and I'm really impressed with your thinking and analysis.
Innovation and marketing (not advertising) are the mothers of successful business; good innovation solves real market problems. It's about giving your customers what they want, not what you think they want. This doesn't mean convincing them they want your product, it means having a customer centric orientation, and bringing true value to their lives.
Google are masters at innovation, but they definitely need to have the real-time web firmly on their radar - and I'm sure they do :)
BTW, this is my first post to SEOmoz. I've landed in this field from a background in film/television and business, and I just wanted to say that I'm loving what I'm seeing. Keep up the great work.
"they did fail to evolve and now are beyond repair." Hold on a tick--are you claiming Google will be beyond repair because it took us ~3 hours to offer autosuggestions related to Michael Jackson? I remember TMZ was out on a limb making the claim and no mainstream news site was able to confirm/deny for quite a while. If it took CNN about 2 hours to confirm the news, it's really not that bad for Google to start offering autosuggestions <1.5 hours after official mainstream media confirmation. (I wrote a couple more paragraphs about our onebox showing ~20 minutes after mainstream confirmation, plus the ability to sort by recent results, etc. It was a great comment. But my comment got cut off, and the pop-up nature of the comment box means that I can't hit the back button to recover/repost the original comment. Also, the comment box still isn't draggable in Chrome--pretty frustrating when you're scrolling up/down to compare timestamps in the post. Anyway, I don't consider Google beyond repair based on the fact that we offered autosuggestions <1.5 hours after official mainstream media confirmation.)
Have you tried typing 'michael jackson is' into google? Apparently the Google Autosuggest thinks he is dead, dying, ill, and back. Not to mention also being Latoya.
I tried this with a couple of other UK figures, like our Prime Minister, Gordon Brown (screenshots here). Apparently he is sh*t, an idiot, useless, incompetent and a pr*ck according to Google Autosuggest.
What's more, Prince Charles is the antichrist apparently.
I'm sure these are all generated algorithmically. But don't you think this needs some looking at?!?
Ha! Thats pretty damn funny! But it sounds like its working fine to me!
That's classic mate, very funny!
It's a tough area for google to try and 'look at' though.. when does making it 'correct' become censorship?
Extremely interesting article.. if only the real-time web wasn't struggling to monetize at the level their investors envisioned, then perhaps Google would be concerned. Things like this give SEOs a window of opportunity to be first to blog or create some content targeting the breaking story/event.
In my opinion Google has several big flaws.. most of which are to thank for sustaining the SEO industry. The crusade against "link manipulation" hinders their ultimate mission to end search spam which I believe could be better approached through education of the overal web user population.
I'll happily take Google in its current misguided form. A flawless Google would mean that most of us would be looking for another way to pay the bills ;)
Brilliant article and something that I think would be fascinating to a lot of people who don't work in the SEO industry as well.
It does not appear to be factual though. From x17online.com: Michael Jackson has passed away at the age of 50, reports TMZ...X17online was the first to report that Jackson was rushed to the UCLA Medical Center earlier this afternoon, and we brought you the first pictures of the pop star as he was wheeled into the hospital.
I don't pretend to know how Google's crawler architecture is setup. But it seems to me that they need to isolate how they crawl news sites, and be able to disseminate news results faster.
I can understand it taking 3 hours to index a post to a popular blog or something. I can understand it taking 3 days to index a new page on my blog. But they sure as hell better get out and blaze breaking news on major outlets and be able to spit it back out for relevant searches in a timely fashion.
When Twitter can fly through hundreds of thousands of Tweets in minutes and update their trending topics accordingly, it makes Google's lag on news items seem utterly silly.
Very nice timeline...
I made a french translation of this post, its worth it!
https://www.jacquesboulogne.com/?p=3002
Wow thanks for doing that! Now more people can read this. Very cool.
Excellent analysis, Danny. I agree somewhat with Timmy & Jeff above that this doesn't necessarily kill Google - they can still dominate regular search and not crack this (for a while). However, against their stated objective of "organising the world's information", this is clearly a shortcoming. I'm sure they are working on some things, but I'm also sure there will be some lucrative aquisitions in this space.
Incidentally, I like this chart of twitter mentions of Michael Jackson: on twist.flaptor.
Danny, Very impressive indeed. Well done.
Personally, I think there's something to be said for letting the facts sink in a bit before presenting the information. If the news (don't get me wrong, I'm not saying Google is a news organization) immediately reported every tip they received, we'd probably all be in a constant state of panic - running around with our hands in the air until facts could be checked.
We all know that Google has different server locations as well...could it be that YOUR google results were slightly behind what I saw? (this is just a thought to consider, I have no idea whether they are or not...) For me, realtimeweb = massive spam opportunity and I don't know if that's really what I want.
All very good points. No disagreement at all.
On a side note I thought this sentence was interesting:
Many would argue this is already the case. I know the local news in Seattle is full of sensationalism and scare tactics. I am not sure about other cities/countries.
For example, this is an actual headline I heard recently: "And at 5, why that headache you have might be cancer". The "story" ended up being about staying hydrated. Sigh...
Has anyone done a Bing vs. Google on this?
Killer job on this. Very interesting timeline - and it certainly illustrates the real-time indexing problem Google is facing.
It doesn't seem impossible to improve speed this - I agree over three hours is ridiculous to respond to such a high gravity topic.
Worse is the near hour and a half it took Google to start showing this story for "Michael Jackson" searches. I have to imagine they can do better than that.
Very interesting article Danny. Thanks for tracking this information and posting it.
However, I think your conclusion might be overstating the case. Yes 3 hours and 17 minutes is a long time for Google to react to something, but that is the duration between a single internet site (TMZ) reporting what might have been a rumor, and Google offering it as an autocomplete suggestion. Is that what we want from Google search? Instant incorporation of possible rumors (or otherwise uncorrobrated stories) as autocomplete suggestions? And is that a fair metric for their real-time search performance?
A better showing for Google was their 20 minute delay from MSNBC.com confirming Jackson's death, and the story showing up in Google News.
That being said, I think your broader point is well made - that Google is not the only or best source of real-time information, and that we, as SEOs, should be aware of the changing dynamics of search in the emerging real-time web.
Interesting article. It's a shame you didn't mention Yahoo! ; they were much faster to get the MJ news than both Google and Bing:
https://www.searchenginejournal.com/michael-jackson-dead-twitter-and-facebook-report-death-before-major-news-media/11386/
Hey Danny,
Thanks for this impressive account of the event and how it unfolded on the media, plus the additional commentary and observations. Thanks for putting so much time on this, I read it all with interest.
Cheers,
David
Brilliant! How could you keep an eye on everything on the Internet during such chaotic time :)
I like the work you did to track how the news spread, but you have to be kidding that this is an issue for google. Based on your timeline it took 20 minutes before it was on google news from it being reported on a mainstream news source. Why would you be searching for Michael Jackson, unless you had already heard that something had gone on. That is really a tiny window from finding out and doing research to find out.
Twitter is not search, if you going to compare then you would have to know more when did Google alerts push out the first sighting of this. That is a better comparison. The technology and needs are completely different. Twitter is a push technology where you can push information, it will do a lot better at spreading a truly news worth story, that is where it shines. Google is where your doing a search, I am sure they could change there results based of a trigger in the search volume on terms, to start to refresh quicker. I would never go to google search for finding out the news. Maybe to point me to a site that has that as relevant. Research information, buying information, navigation searches .. next to no searches need upto the minutes information. The only possible thing is news and even then something like radio or tv of a live feed getting information as it happens is probably a better bet.
I think you are to hard on Google.
You have to look at it from the search engine perspective. I agree with 'Karmakeys, that when Google puts 'news' to the top position the second they find it, there would be a lot of fake news and rumors on the front page. That would ruin their reputation. People rely on Google as a source of truth. If Google publishes it on the front page, it must be true.
A factor not to forget is that a news site has human jugdement to decide if a news fact is true or not. Google hasn't. An algoritme must decide about the reliability of the fact published on some site. Possibly the best way to verify the reliability of news, is to build in a delay and wait to see if the news persists.
What blows my mind is how CNN seems to still be the key MSM component in the "emerging real-time web." Although MSNBC did beat 'em to the "death confirmation" punch, CNN was positioned to outpace MSNBC.
This points out MSM's role in the emerging real-time web, and what they can do to shore up what they need to, er, shore up. I'll wake 'em up and tell 'em so.
VERY interesting. How did you get all these times exactly? It was a great test. Be interesting to see if the search engines learned anything from it.
I've read a few articles on this subject. And this is a fine one. But a couple questions. One, we should we expect a search engine to be able to deliver news results immediately? And two, why do people even USE a search engine for breaking news?
I was in my truck when I heard a reference to Michael Jackson's death on the radio. I called into the office and had a co-worker check online. She popped through several "news" sites including CNN and Yahoo among others. They had stories about the cardiac arrest but nothing about his death. I made it home and was popping by several other online news sources and was still getting the same. I was beginning to thing that what I'd heard on the radio was in error. Or that I had misheard the radio. (It was on as I was getting in and starting my truck so there was the chance that I hadn't heard the entire story and I had misinterpreted what I had heard.) Took my son to tae kwon do, came home and looked again. This time I indeed found confirmation online about his death. More than 30 minutes after I'd already heard it on the radio.
But out of all of that, it never crossed my mind to "search" for that news.
It sounds like we approach news and search engines very differently. I wrote about this more at this comment.
Did you happen to also record the timeline when Harrison Ford and Jeff Goldblum died?
Those two didn't die, it was a hoax.
I think that was the point.
Google's relative slowness to incorporate these types of stories into its news results probably helps keep hoaxes and rumors from being picked up and propagated as legitimate news.
Glad someone got it.
JUST BRILLIANT!! Very good article.
Just one question how to know when an article is indeed a real fact. Going into cardiac arrest is one thing but dying is a very different thing.
What if an article stating the fact of a persons death is faulty....still Google and other sources might index it doesn't it? I just can't imagine the impact something like this might have.
BTW a slight advance on what this Seomoz.org project might be would be nice....
In my humble opinion, I don't think accuracy is the most important goal for Google. See https://www.seomoz.org/blog/a-bad-day-for-search-engines-how-news-of-michael-jacksons-death-traveled-across-the-web#jtc89377
Fascinating article! A lot of people are bashing Google for lagging behind in the real-time web market, but no one has even mentioned Google Wave yet. They showed it off a few weeks ago and it's all about live collaboration on the creation and dissemination of content and information. (Great, now I'm talking like Jesse Jackson!)
Anyway, my point is that it's not like they're just sitting around wondering what to do.
Wow I have never heard of this before, what an interesting spiritual story. It's truly an inspirational and interesting story.
O.K. So I'm an old loser from the MSM, but sometimes, dammit, it takes time to confirm stuff. And since bad news travels way much faster than good (see Romeo and Juliet or Orson Welles' War of The Worlds--yes, I am that old) your big guns, that people rely on, move with some caution. And Google is one of those big guns. TMZ is not the TiMeZ, and X17 is a bunch of paparazzi with expensive police scanners. So maybe it's not a TERRIBLE thing that Google waits a beat or two before adding "in the water system" to its search for "Anthrax," for example...
"So maybe it's not a TERRIBLE thing that Google waits a beat or two before adding "in the water system" to its search for "Anthrax," for example" Good point.
I think our point of contention is what we believe the purpose of Google should be. It seems like you rely on it for accuracy (correct me if I am wrong, I don't want to put words in your mouth) and I rely on it for relevancy.
It is very clear to me that three hours after one of the biggest music celebrities in the world dies, a search for his name should return information on his status rather than on one of his albums. At that point in time the most relevant result should be the website millions of people are talking about and searching for.
As far as accuracy, Google has never claimed to be accurate, it only reports what it finds relevant on the internet. For example, do a search for any major prescription drug that people want without prescription. Are you going to find 10 results from trusted doctors' websites with accurate information? No you won't. Instead you will find what Google thinks is most relevant to your query. (On a different note, this is actually an area where Bing does really well)
Your thoughts?
Sorry to take so long to reply. (Have a job, luckily, and would like to keep it.) You're quite right of course. Google was frustrating that day. Google is increasingly frustrating and returns irrelevant results. My point is, in this privileging of speed over all else, which your fascinating article seems to do, there be dragons. If you search for MJ and the top return is NOT that he's dead, it's because, in my assessment, it's because a whole bunch of other outlets I trust have not called it. I rely on google for not accuracy so much as acceptance. If Google is returning a lot of stories from reliable outfits that MJ is dead, then it's more likely he is than if just TMZ is. It was a bad day at the Goog, but it would have been a worse day if it had gone the other way.
Oops I did it again. Just now. (Used Twitter as my go-to search tool on a breaking story. Before Google.) Shhhh, don't tell anybody.
That has been happening more and more. This is a great example of my high-level point.
Great piece. I cited your blog in my article at the Examiner. See here:
https://www.examiner.com/examiner/x-10080-DC-EGovernment-Examiner~y2009m6d28-How-Michael-Jacksons-death-made-search-engines-go-crazy
Tanya
Google has been aware of this problem the moment Serg announced he was gonna work on the engine again.
True that this event kicked Google's search engine in the balls.
But obviously this just a small problem that the team is looking to address.
Google rarely disappoints and I'm wondering what they are going to release a year from now.
They realized the need for real time search with Twitter. They already countered it with their added search options back in May, everyone/anyone using Google can do a recent search and get real time results.
Fantastic post! I think this will be a post the will show where search was in 2009 as a case study for years to come. Excellent resource to show clients how information spreads.
Well said Sarah! That is true. This clearly shows the information flow and how Google has been influenced with that information.
Ok, but I wonder much time went by before Google returned a relevant result for a search for "michael jackson died" or "michael jackson cardiac arrest/heart attack".
Additionally, I don't think using the auto suggest freature as a measurement of how quickly Google picks it up is very accurate.
Finally, I don't think a general search for "michael jackson" would be the most accurate search term for this result. I think "michael jackson died/dead/death" or "michael jackson cardiac arrest/heart attack" may have provided faster/more accurate resutls.
Do you have any info on that?
I disagree,
I think a link to the TMZ article would be the most relevent result for a query for "Michael Jackson" given the timeframe and context. (Relevency is not the same as accuracy). That is exactly why Google inserts Google News at the top of SERPs.
Interesting article but just a correction: MJ was not at Neverland Ranch he was at his rented Mansion in Bel Air.
Thanks! I appreciate the correction.
Thanks for this great summary. Its a really sad occasion - but two days ago i finally realized, that twitter brought us to new era of the internet. Combined with the unrests in Iran, it shows how news can spread dramatically faster. Here in Germany it was about midnight and all news agencies on tv or web were literally sleeping but i was able to catch up news in real time via twitter.
So i totally agree with your post and hope more people realize the importance of the "real time" web and google can soon manage this new flood of infomartion.
>> arrived at Neverland Ranch
Um, you see, that is exactly why there are so many rumors still floating out there around on the internet, and how many people got it all wrong. Michael Jackson hasn't been at Neverland Ranch for a long time, now, he was renting an estate for $100,000 a month. I hate to be harsh, but you need to do a little fact-checking before you post.
It wasn't Neverland Ranch, a simple check of Wikipedia will show you that MJ hasn't been there since at least 2007.
>>UPDATE: glad to see that you've update the post with the correct information.
I have to agree, as well, that a delay of 3 hours and 17 minutes is not ideal but it's pretty darn good. Just remember where we were a few years ago, when Google took more than 24 hours to update its index properly? I do think that the time will come where we're only minutes behind in indexing.
The only way I can see information being update much quicker, as in the real time web, is if Google starts relying more on our browsing habits and what we're searching for and what URLs we're on right now. Or maybe they're already doing that?
You just provided a perfect example to prove my point. I had updated the post 4 minutes before you posted your comment. That shows the importance of a true real time web. The information you got when you loaded the page was obsolete by the time you finished reading the post. Crazy world we live in. (Don't get me wrong, I appreciate that you took the time to correct me. I did indeed screw up and I deserved to be called out. Thank you!)
Bravo Danny, Sometimes its easy to forget that even the most ingrained titans of the industry can be unseated. (myspace-facebook, facebook-...google?). This might have been the decade of Google, but which company's strategy will provide the user framework for the next decade of the internet? Its impossible to tell.
Yes, when the usual processor in the world be AMD or Intel 20 TeraHertz, this might be possible
Probably one of my favorite SEOMoz articles. Extremely interesting! Thanks!
This really paints a lush tapestry of media and how it unfolds and intertwines as events transpire. One can visually see an explosion in communication. Very great post.
Very interesting article. Thanks for posting this!
A couple of things though. I don't understand why you place so much importance on the suggested search element of Google. If someone wants to find out about his heart attack, surely they're going to type that in. If they're not looking specifically for it, then no big deal that it doesn't arrive for 3 hours.
Also the relying on Google for your incomes part... what kind of business are you in where you need instant updates on your products? I know getting there first can be a huge advantage in the modern online era, but when has 3 hours ever made a difference? You say it puts food on the table of your families, which I'm sure it does. But that's a little melodrmatic isn't it? A little too emotive counjouring up the image of your kids, in rags, hungry because google took 3 hours to update that MJ was dead?
"But that's a little melodrmatic isn't it?"
Yes, I should have worded that more professionally. I took that a little too far :-p Duly noted for next time.
"when has 3 hours ever made a difference?"
Maybe this is just a side effect of my generation and profession but I have learned to expect quick results. Instant Oatmeal, Instant Messenging, Instant results. I mean this in the most polite way possible but if you don't care about the value of three hours, I will gladly compete with you in business.
I will bet on the value of saving people's time every time.
Thanks for opening our eyes to these new realities. Being a PRO member I eagerly look forward to SEOMoz real-time web to see how it benefits us.
Fantastic. Thanks! Really good. Its also a very good illustration of how information travels around the web!
BFD You say this window is dragible - NOT. Had to change screen resolution to proceed, For news, go to news. Don't use a search engine that is monitoring 56B websites. JW
Great job with the timeline and post! It was very interesting to see how long it took Google to start recognizing the trend.
Looking forward to the real-time project you guys are working on.
I did something similar, checking the SEO Results for the term "Michael Jackson Dead" on the same day.
Surprisingly old listings from 2003 still ranked in the top 3 and news ranked highest, particularly articles outside the US, like South Africa and Australia...interesting!
Thanks for such an interesting article. I guess it takes a big event such as this to make you realise just how slow Google can be when it comes to real time news. 3 hours and 17 minutes is indeed a long time on the net and people will go elsewhere. It is true Google's technology isn't flawless but in the majority of my work - working with small and large business - it does the job.
This real-time web sounds very interesting though.
R.I.P Michael Jackson!
This post is incredible, well researched and hugely informative. I look forward to hearing about your plan for real-time web search.
Awesome to use such a world-spreading news items as a case study, definite thumbs up!
I think it should be noted though, before any preemptive calls of various news sources deaths, that the internet is not a wholly accurate source.
Yes, TMZ and co got there first on the death of MJ, but part of the reason why so many people searched on the web was that they weren't convinced it was true. Offline media sources report on facts alone and only allude to the rumours, so are significantly more reliable.
Twitter might be rapid, but did anyone else note the other day when Rick Astley's death was one of the top trends? Not just him either - Jeff Goldblum, Harrison Ford and George Clooney have all apparently died and reincarnated themselves since MJ expired if you believe some gossip sites and Twitter services.
In many ways it shows how reliable Google is that they waited until the reports were confirmed rather than post on what might have been scurrilous rumours. I say fair play for trying to maintain their accuracy in a world that often lacks it!
You didn't include that the MSM then picks up the story well after Google, and then ties up the media channel for hours bringing real-time coverage of the non-news of the event.
Filmstrip demonstrating: https://picturesforsadchildren.com/blog/famous.png
:)
This makes me wonder if Google is "spreading themselves too thin" having their hands in so many projects. Nicely written Article BTW!
I highly do not agree that Google has themselves spread to thin. They have a rather larger staff in multiple locations. Sydney is spearheading Wave, they have a development team for the Android, they have teams for search engine help/aid/development, they have various teams for Google Maps, etc.
I think Google's success right now is the fact they are handling so many projects and they seem to be nailing every single one.
I saw an article in the Official Google Blog stating that searches for 'michael jackson' started rising fast around 15:00 PDT, which is 22:00 GMT. The rise in the number of queries is an important factor in the determination if 'new information' is really 'news'. With the timeframe for the number of queries in mind I think that Google responded very fast.
See: https://googleblog.blogspot.com/2009/06/outpouring-of-searches-for-late-michael.html
A well-done article is placed here. It's amazing how the interenet was bumped up by the requests of users who rushed into search engines, social-media, and news sites to get update and stats show how it was over loaded.
I've one technical note for the article itself: is it possible to add the timezone to the mentioned time? I think this can be good reference for the whole story.
Thanks! All times are in GMT time which is supposed to be universal time. I did this precisely so time zones were not an issue. Does that make sense?
With a few good posts like this I dont think it wont take Google to long to react. And it is true that we all hold Google to a higher standard and we do expect more.
Great post! I was hoping someone would cover this from the SEO/SEM sector!
As an ongoing student of SEM and marketing trends I was blown away by yesterday's events. I was floored that I spent much of my afternoon watching live feed from the hospital on TMZ. I'm still amazed by how powerful that site is, especially considering Google, facebook, and twitter were all taken down by the MJ news.
Glad you guys are forward thinking and planning for the real time web! Can't wait to see it!
Hi Danny,
Great post, I was thinking about a post such as this on the night itself and was amazed more at the spread of the story on Twitter and Facebook.
Can I ask how you managed to get the approx times on these events? I'm assuming you were watching them unfold yourself?! From an SEO point of view it would be nice to do a report on a linkbait story for example and show a client how their story unfolded online.
Paddy
Paddy,
As I watched this story unfold online, I opened new windows as the story spread between websites. I then continually did refreshes. Once I noticed a major change, I took a screen shot.
After I was done, i used the created time of the screen shot (as assigned by my computer's operating system) and the information they documented to piece together the time line. (My computer's system time is set to sync with the atomic clock just like most Macs)
I mention that some times might be off by 5 minutes because it is very difficult to pinpoint the time when major websites go down. As I am sure you know, they use load balancers and content distribution networks to distribute information. This means different computers will get different results depending on where and when they request the page.
Yes I understand what you mean, thanks again for the great post and reply Danny.
Nice & precise point. Google's perfection will start by admitting that they are not flawless.
Indeed, Google has became the bread & butter by internet marketers. Updates on information like this is a must-know for SEOs.
Awesome article. I'm amazed at the amount of research that was needed to pull this off.
What about search engines like https://www.collecta.com/? It updates in realtime as it finds content using xmpp (same thing gtalk and a lot of other instant messengers are based on). https://www.washingtonpost.com/wp-dyn/content/article/2009/06/19/AR2009061901228.html is where I first found the story.
Also, I know this is off topic but as a developer I had to go through 2 browsers to be able to post my comment. The div that is "draggable" doesn't really work to well in ie or chrome. It places the "post comment" button just below the fold and you can't scroll to view it (at least for me). I see your using scriptalicious which is awesome but adding a handle to drag from would help a lot.
Code looks something like this:
$(chatwindow).dragable = new Draggable(chatwindow,{zindex:99,handle:'chanhead'});
See? You can use a "handle" so the drag event will fire correctly (mine works in ie and chrome).
Here's the ref docs for the draggable -
https://wiki.github.com/madrobby/scriptaculous/draggable
Thanks for your time sorry to complain but i had some trouble adding my comment.
Did you do a recent search of Michael Jackson as it was going on? I was searching on the show options -> recent search and I was getting constant new news feeds as they were posted. It seems this study was just using Google's basic search...
Valid point,
I chose basic search because it is Google's most used feature. If you really want to make an impact, you need to appear on the homepage, not four clicks away.
But then IMO Google isn't lacking real time search at all. It would be like using a bag that is 4x too big for your needs vs. a bag that perfectly fits your needs. Instead of saying Google is lacking real time search, I think a better approach that Google needs to find a way to make the new search options such as recent search/wonder wheel, etc. more commonly known.
This is a very interest post which explained the message spreading very well. I think it's time for google to come up with a filter to identify whether the high traffic is an attack or just an explosive news.
I am sorry, where is Google's flaw again?
That they didn't offer "Michael Jackson" query automatically on homepage?
That it only appeared 3h 17 min AFTER the event?
I'm sorry but since then do you SEARCH the news on Google immediately after they happen?
I am sorry but what you are saying is downright stupid and wrong about Google. They did their job, the outage was really short. And what is the emergency for you to get this as an auto query on Google?
Oh, you are so damn important customer to Google of course, that you went the other direction. I guess t some news site. So, guess what - that's where you had to start in the first place.
It's just that Google is so great that you're addicted to it to get every piece of your information daily to go there. "Just Google it" - but Google is not the universal solution.
PS Describe any real-life situation where a customer walks away from a SEARCH ENGINE (because this is what you're talking about specifically) only because it doesn't show an auto query?
And how this affects you as a webmaster (supposedly a customer too), if anyway you'll be found through other spiders?