If a human from 200 years ago were to look at our planet today, it would appear completely alien. Culture shifts and technology improvements have drastically changed our perceived landscape. One tiny part of this is the use of social media and search engines. More than ever, teenagers are complaining about their parents joining Facebook and parents are complaining about their children interfering with their online social lives. Together they are sharing small events like sore throats, runny noses and big events like floods and hurricanes. Communicating online has become intertwined with our lives and has now become deeply integrated with our work, education and entertainment.
But how did this happen? While I don't know for sure, I do recognize the patterns. For companies like Twitter, Google, Digg and Facebook, it started with a small group of entrepreneurs in California whose great ideas eventually went viral and spread around the globe. This word ‘viral’ describes a pattern and has become a buzzword. It is usually used to describe the virus like spread of ideas and technologies. The amazing idea behind a virus like spread is it expands exponentially. Once it starts, it multiples and multiples until nothing can stop it.
It is the great irony I see in this buzzward that prompted this post. I believe the viral nature of social media and popular technology companies is what will paradoxically allow us to prevent the viral spread of real viruses and pandemics. This is not a new idea. Many vaccines are in essence inert viruses fighting would-be viruses.
In 2006, a man by the name of Dr. Larry Brilliant won the TED prize for calling for a new global system that could identify pandemics before they spread. Dr. Brilliant (you can’t make this stuff up) is world renowned for his efforts in successfully eradicating smallpox from the planet. Before winning the TED prize, he had been inspired by the potential of a Canadian system called GPHIN. GPHIN is a system of web crawlers and analyzers that scour web based content looking for trends in keywords like ‘fever’, ‘cough’, ‘tired’, ‘sick’ and ‘flu’. Using this methodology, GPHIN was able to detect a would-be SARS outbreak six weeks before any other system (including the systems used by the World Health Organization). Quick responding officials were able to isolate the outbreak and prevent a global pandemic. Dr. Brilliant later said that this possible for two reasons.
- Early Detection
- Rapid Response
Unfortunately, this is not where the story ends. Larry Brilliant has since left Google for other endeavors. GPHIN and Google Flu Trends continue to save lives but they are only the tip of the iceberg. Google is now falling behind new competitors that dominate the ever growing real-time web. Status updates from Twitter and Facebook are being produced and becoming obsolete so quickly that Google’s index can’t keep up. We saw this with Michael Jackson’s death and we will see it again in the future.
Social media has the potential to become the greatest early detection system that the world has ever seen. It is faster, nimbler and has more access to user data than any traditional search engine. Not only does Facebook have the data necessary to see who is suffering from an illness, it has the data necessary to predict who these ill people will most likely come in contact with. Twitter has the data to make similar predictions (although less accurately because people don’t physically spend time with Twitter friends like they do with Facebook friends) but enjoys the added benefit of being accessed and updated from any place with mobile phone or WIFI service. (90% of Twitter requests are made to it's API, whereas only 12% of Facebook users access Facebook through it's mobile apps).
These two social media platforms by themselves have the ability to enable ordinary people to report their symptoms in real-time. Specialists like epidemiologists and statisticians could then identify threats (early detection) and use these same communication channels to direct aid workers (rapid response) on how best to isolate viruses before they become pandemics. If the features of other social media platforms and modern search engines were added to this theoretical system, specialists could for the first time ever educate the global community in real time. (Think about how many people read stories on Digg or about the much larger amount of people who read Google Adsense ads every day.)
Theory and predictions are helpful but just like verbal contracts, they are only worth the paper they are written on. Luckily for us, this theoretical system is already becoming reality today. People are already reporting their symptoms on Twitter and on Facebook. Likewise, disease experts and aid workers are already using social media to organize relief efforts. Although a unified, non-government controlled system for monitoring these platforms doesn't currently exist, all of the pieces are in place. Dr. Brilliant said that there are two steps necessary for preventing pandemics. Social media is completing the first step (early detection) to a degree that even he couldn't imagine. Better still, this is not costing the public a dime.
The world is changing in parallel with the internet. Next time you hear someone complain about "pointless" status updates, take the time to explain it to them. Social media is powered by all of us individually. Because of this, you have the ability to make a positive difference.
UPDATE: There has been some very healthy discussion in the comments that I would like to address here. Thank you 0lly and Bludge for bringing this up.
What about false alarms? Clearly, social media will have a lot of noise mixed in with legitimate concerns. Just like good ideas, bad ideas can spread virally as well. Paranoia and misinformation run rampant in the media and online.
So how do medical professionals use a system that will likely have a lot of misinformation and noise? The key is acknowledging that limitation and designing the system to account for it. The reason that GPHIN was so successful, was not that it sent an e-mail directly to health authorities every time someone mentioned "cough", instead it was so powerful because it used aggregate information to identify real trends. Real people, including doctors, look over the information and decide if it is worth pursuing or not. It is natural to assume the front end of a system like this might look something Twitter search with has every relevant Tweet visible. This is not how it works. Instead I think a more accurate representation would be that of Google Flu Trends (aggregate information) with trends that can be broken down by location and with the help of social media, broken down into social groups.
I am not and do not claim to be a medical or disease expert. If you are, I invite you to help me make this post better. As always, feel free to leave your thoughts in the comments below. If you would rather not do that, feel free to e-mail me. All of my contact information is available on my profile: Danny Thanks!
Other Similar Discussions:
Twitter: Growing Virally But Can It Stop Viruses? - Chris Thorman writes a very compelling post that adds the use of Electronic Medical Records (EMRs) to the discussion. From the article "The combination of social media and EMRs, in some form or another, will undoubtedly be part of the future of tracking disease outbreaks. The how and when of that process remains complicated, dependent on health agencies, governments and the doctors themselves to implement the appropriate systems."
Okay, brilliant stuff Danny.
And here I was thinking most of the communication on Twitter was just crap-o-la. The thought that it could save us all from a terrible death is intriguing. Can it save us from anything else. (politicians, radar traps, visiting relatives?)
Just asking.
lol, time will tell but I am not hopeful.
If anything, social media is making interacting with relatives worse. No mom, I don't want to join your mob war... ;-p
I can see that these flu trends, and news about a pandemic cause panic in some people, but I can't see that save any lives. Its certainly interesting to see these numbers, but shouldn't pharmacies, or, even better, health professionals, have much better numbers? How many people are out there and said they have the swine flu, just because they think they're funny, or they think its a trendy thing to say?
Social Media might show some trends, but one can't rely on these numbers, as the masses are easy to influence.
That's part of my point. The numbers calculated via systems like GPHIN were the best numbers. That's why it was able to detect that SARS outbreak 6 weeks before anyone else. Even the World Health Organization couldn't do better. Social media has the potential to trump all of this.
In fact, you don't have to take my word for it...
According to Google:
https://google-au.blogspot.com/2009/06/google-monitors-influenza-trends-in.html
With regards to false alarms, that is inherent to a lot of systems and the designers take actions to minimize their affects. Obviously, they are not putting the emphasis on people who flat out say "I have Bird flu". They are a bit more creative than that ;-p
For any individual, that's very true, but on a massive scale, much of that works out in the wash, mathematically speaking. We've seen a huge growth in prediction markets over the past few years, for example (thanks to books like "Wisdom of the Crowd"), and it's amazing how often this kind of mass-data is more accurate than the experts. Experts have their biases, too.
At the risk of incurring innumerable thumbs down for having the audacity to openly criticise staff posts, I think this is not only fanciful but potentially dangerous. Please allow me to explain before you all stab the thumbs down button.
Now I am all for early detection, and I really am amazed by the speed information can travel around on the various social media networks, but the quality of information is, lets me honest, often not of the highest quality. I’m sure you all read the report about the study of the quality of Twitter posts the other day (https://news.bbc.co.uk/1/hi/technology/8204842.stm for example). Obviously this is only a snap-shot study and slightly tongue-in-cheek but we all know how much pointless “noise” there is.
Now you are seriously suggesting that the offhand musing about how someone feels on a particular day can for the basis of a medical diagnosis? Are you insane? In the UK, the Swine Flu epidemic has been dealt with largely by a phone system and many experts have suggested there have been a huge amount of misdiagnosed cases because of the difficulty of diagnosing people in this way. It would be harder in text form, not to mention the fact most people are not medically trained, they will not understand their symptoms when they are/or are not ill. But they’ll Twitter anyway, because, what the hey, everyone posts everything on Twitter regardless of quality.
This is a well written article, but in my opinion, it is wildly optimistic at best, but a potentially dangerous route to go down. Twitter users and statisticians are not medical professionals, you are likely to cause panic more than anything else.
“Drowning in data, starved of information.”
I believe researchers are looking for patterns among large groups of people. And naturally they will include other data as well. I see social media as just one more tool in their arsenal.
Love this post, Danny!
I was actually going to comment on the Swine Flu. I truly believe that social media is better suited for rapidly spreading news (Iran, Michael Jackson, etc) rather than diagnosing an epidemic. Bludge was right, it is next to impossible to predict if someone is being totally accurate when they say something like "My stomach hurts, it feels like a grenade went off in my stomach". Medical software won't be able to pair that with someone else being as dramatic across the world and alert the authorities that there is an uprising epidemic. Lets also remember that epidemics (Swine Flu) typically all come from the same local region. The Swine Flue virus started in Mexico. Sure it spread with tourists. But, it didn't just start randomly with people all across the world Tweeting "I'm feeling sick today".
Of course data from social media will be noisy on an individual level. The trick is sorting the signal from the noise, and there are plenty of sophisticated data analysis algorithms for doing that.
You are right that any particular post about medical symptoms is unreliable. However, you would expect the background noise of unreliable reports to be relatively smooth. If you see a sudden spike in reports of stomachache that is statistically significant and well above the normal background noise, you might reasonably interpret that to mean something real is going on.
(added)
As anybody with significant traffic to their website knows, the behavior of individual humans is erratic, but en masse they are remarkably predictable.
Also, with regards to your link, I would think that much of the information useful in this sense would fall into the "pointless babble" category. Their example of "pointless babble" was "I'm eating a sandwich." The authors of the study might consider that pointless, but I'm sure people in the food industry would find it very interesting. Likewise they might consider "I'm feeling like crap today" to be pointless, but in this context it would be highly relevant. One man's trash...
Didn't thumb 'ya down Bludge. But did want to add my 2¢.
While I'm not a power user of twitter due to time constraints, I can see how it could have great benefit if monitored by those with resources at their disposal. And not only for medical issues.
It's a great way to get the "pulse" of a situation, provided you are able to sort the chaff from the wheat.
Hi Buldge,
First off, I want to say that you and everyone else should feel free to question everything I write. I hope you do it constructively (which you did) but that is not required by any means. I have gained a lot in my life because I try to question almost everything I read and hear. (Just ask other mozzers ;-p) There is a lot of misinformation out there, it is up to each of us to decide what is valuable.
Secondly, you bring up some very good points about noise. I met with a virus specialist at the University of Washington Medical Center today and asked him about your question.
He described the system we use in the United States for cutting through the noise as the following:
General practitioners are the front lines of early detection and serve as surveillance. Whenever they see anything that looks suspicious (even if they just think it is Strep Throat) they send a sample to the city health department for more tests. If they see a problem they then send it to the Center for Disease Control.
The problem with this system, is while it is highly accurate, it is very slow. It does act as a solid backbone but other, faster tools are necessary to monitor things as complicated as public health. This, in my opinion, is where the system I described fits in.
Thoughts? Am I missing something (again)?
:-)
I was talking last Friday with a guy (not in our industry) that had a prediction about where he thought Search Engines were headed as far as technology he thought they should be working on.
He felt that the individual webmasters should be able to put an icon that the search engines would recognize on their page to help sort out what words that page is designed to rank for and that would solve the issue of irrelevant results. I know what you're thinking but stop.
I know what he came up with is something we would all recognize as another opportunity to falsely manipulate serps, but I give him credit for at least thinking outside the box.
Not everything that everybody comes up with is a great idea, however it can be the beginning of something great down the road.
I think Social Media has a powerful future. It isn't there yet. But if people keep trying things like Danny wrote about, who knows what the future holds.
Remember, if we had computers in the 1800's, they would have predicted that every city in the USA would be thirty feet deep in horse manure by 1950. :p
Love that last point :)
Forecasting is a tricky beast!
Danny,Thanks for replying. Firstly, I was being slightly melodramatic about “daring to criticise staff” which didn’t add any value to my point. Silly me.Anyway, I understand your enthusiasm for the uses of social media. I’m an SEO (only one years experience) and as I say am continually in awe at the speeds of information the social networks can provide. I can also appreciate that as society becomes increasingly mobile disease has the potential to spread rapidly, but that’s all the common ground I can grant you!Social media is social. It is a place for wonderfully inane nonsense to be combined with the transfer of the latest and most news and events, whether that be of a personal nature or international importance. But people interact in unique ways, there is no standard way to communicate, no way of measuring for consistency in what people say, and no basis in fact. It is at the far opposite end to “scientific” as you can get, which is crucial in medicine. We can’t work on unsubstantiated hunches. I understand the point someone made about “if enough people say it there must be some significance”, but it’s Twitter, beautifully anonymous and no way to base any kind of medical assumption.Anyway, others have made the point more eloquently than me, but thanks for the debate. I do love a good debate! I must go now as I have a bit of a stomach ache... ;-)
Danny, very interesting post. Google needs to get their hand on the real time data faster. It will be interesting to see what they come up with for the future, but will it be too late?
Edit: I have fat fingers tonight and can't spell! =)
They seem to be getting quicker and quicker as time goes on. I have noticed my blog posts getting cached in less than a minute from the time that I post them.
Also, lets not forget about Google Caffeine
That's good to hear. I am hopeful that they will be able to improve. Google is made up of an amazing group of people. If anyone can do it, it is them.
Danny, This was well-written. I tend to be a little cynical of Twitter. This post allowed me to see it’s potential from a completely selfless point of view. Thank you.
Really nice post, Danny
While I am reading this post, I am thinking about social media may destroy the world as well.
It's all about viral spreading, imagine if some evil organisation started to spread a message of incoming terrorist attack, everyone goes panic.
Maybe I am thinking too much, lol, thank again for this well-written article.
Good point, it could go that way too. We have already seen signs of this with Bird Flu and fake celebrity deaths.
Danny,
Great post - and very thought provoking. Certainly these patterns have the ability to go beyond early detection of disease - and into other pattern detection as well. Clay Shirky talked about this a little in his recent Ted Talk (mostly about events)... I'm aware of one company already that is working on taking the Twitter "firehose" api feed and working on pattern recognition for advertising companies... Thanks for the great thoughts...
This is what I like about SEOmoz. 48 comments on a flu post. :-) You have to love this community. Thanks to the SEOmoz team to make this happening over and over again.
Great post Danny! It's really interesting to see how SM has evolved and what kind of 'social' good we can extract from it.
Although there are technological barriers I can see many opportunities for tracking/monitoring and potentially even forecasting a wide variety of topics like this in the future.
I think as a collective society as a singular consciousness will inevitably guide itself in the right direction. The use of technology and social media allows us to see a live real time representation of the world. Demographics are available offering more information to filter out noise in real time.
if 23 year old males in Germany all started to sneeze????? We'd Know!!!!
Love the Post Danny.
Was the post your brain child?
It was for about 36 hours until I completed it. They grow up so fast these days ;-p
I have one more post similar in style to this cooking in the back of my mind. I probably won't post it here, but keep an eye out on @DannyDover in the next couple of weeks for a post on privacy and the internet generation.
I think its important for Search Engines to be very careful how much "recent" or "instant" searches it includes in the results or it needs to be very selective on the key terms they use to show/generate instant results much like the parameters that go along with local search. I search 'movie theater' and poof local results, if I search 'celebrity death' poof most recent results (example how instant/recent results could work effectively; but if I search 'flu' I am not sure I care Grandma Jelly Bean has the flu in Peoria.
One thing this post does show is that the vast amounts of personal data that can be drawn from Twitter, Blog, Facebook, Myspace and other social media platforms which to think someone could use this information for ill intentions is scary.
But I find it interesting how we can predict flu out breaks but a guy writes on his blog in Pittsburgh that he is going to kill a large number of people weeks before it happens and no one stops this loon.
Thanks for the post very interesting information.
It might be because the guy in Pittsburg did not have his site optimized properly.
Maybe if he had hired an SEO instead of trying to do everything himself...
Maybe more people writing about the same topic would have tipped off the loon-o-meter.
Or if he would have just used Twitter to point at his blog, maybe then. ;P
SEO for murder, death, kill--sounds dark.
I've gotta point you to a friend of mine, a Disaster Sociologist who is actually studying and working with many of these social networking tie-ins to disasters.
https://www.jeannettesutton.com/Papers_and_presentations.html
Highly recommended. Almost 2 years ago I had a discussion with her to help her understand the technology side, but I take no credit for the amazing research she's done in this field.
Facinating post Danny!
I guess this won't be the end though; in 20 years there'll probably be an iPhone App which monitors your breath and skin for early signs of disease and aggregates the data.
In 200 years we'll have a wireless chip in our brain that (amongst other things) automatically pings Google every time we sneeze.
Also: a link to TED at this time of the morning? How am I supposed to get anything done for the rest of the day now? (It's the thinking man's TV Tropes.)
What a timely post. I find myself reading this as I switch back and forth between SEOmoz and Twitter, searching the first bits of news around the tornadoes that hit the Twin Cities this afternoon. It's crazy how quickly news, pictures and warnings spread through social media.
Maybe Twitter needs to hire some Twends analysts to monitor and verify this stuff--sort of an early Twedection system.
Great article, thank you for applying real time social media networks to tracking global "pan" and "epi" demics.
With the shifting of the growing regions we are seeing current environment conditions change, thus, in terms of plant life, molds, diseases, fungi, bugs, either propagate faster or die off. In the coming years it will be imperative to monitor these changes to monitor the outbreaks of damaging molds and plant diseases, or predator species were none were before, and the loss, or growth, of plants and animals. I believe that by reporting were and when various examples of these "nature metrics" appear and dissappear in real time we can begin to track and understand the ebb and flow of natural disease and pest and the existance of species variations in a region.
I.E. A real time geographical map of the spread of the Pine Bark Beetle currently erasing woodlands in the West would help understand the infestation.
Anyway-Back to cold calling, urgh.
That is fascinating, about the early detection system.
You should Google 'Veratect' under Google News.
Bottom line- all the data in the world does not equal recognition, warning, and actionability. Near-real time or not is irrelevant.
What is ultimately required is subject matter expertise who understand how to handle the information, interprete the signal (and associated false positives), and properly trigger a warning. Without this, you get hype and noise in the system that can rapidly desensitize the public... do this enough times and no one responds when the need is actually there to react fast.
The key issue here is data overload and relevance of the information.
Ok 1st off, everyone says Google is falling behind on real time search, guess what, they aren't really. Google just needs to make it more well known that under more options, there is a search for recent results. This came up on the Michael Jackson blog post that Danny did, I'll say it once and I'll say it again, if you use Google's recent search you get searches that could have happened seconds from your search. During Michael Jackson's death I was getting new and different articles by the minute of what various news outlets were saying on the matter. Google isn't falling behind, they just have options that aren't being publicized.
2nd off, Twitter helps spread a lot of solid information, in California we had an amber alert and the I-5 came up as a common trend because people kept re-tweeting an amber alert on the I-5 be on the look out, etc. If Twitter does in fact win the Nobel Piece Prize it was nominated for, then Twitter will see a large increase of users I am sure.
3rd off, If a program like this would become known and wide spread, then their will be some major false alarms. I know Danny said the post their would be some false alarms, their will be some "attacks" if you will that may look completely real. 4chan for example, a lot of them are in different places, a lot of them could use proxies, I am pretty sure if a flu program such as this that 4chan would make some major false information go around. However, the overall good a program like this can do, outweighs the fact that every now and then some people will try to alter it. Just know the good comes with the bad
Great Post Danny. It will be curious to see to what extent technology will addapted and developed to help with disease prevention in the future.
Great Post Danny, Social media does have the potential to become the greatest early detection system that the world has ever seen.
Is this detection "system" already in place? And if it appears to be "succesful", why has it not picked up Swine Flu, which is now currently been flagged as a pandemic. Surely you would have mentioned this as one of your examples in this post, as it is the most recent pandemic to have swept parts of the globe. do you have any information on social media and Swine Flu?
As I mentioned in the post, all of the pieces are in place for this system but its had not been created. Similar systems exist (GPHIN, Google Flu Trends) but as far my research showed, they were not using social media yet.
... These systems did not issue early warning of the current pandemic.
There are no cures or vaccainations for the H1N1 flu, keep that in mind. Flu's in themselves aren't exactly curable, that is why every year you have to get a flu shot, and sometimes it doesn't work.
However knowledge on how to protect yourself, and knowledge of the flu can be spread. Which it was, when it first was announced as a potentially dangerous strain of flu, it was all over the Twitter as a common trend, information regarding it spread like a wild fire.
You can't stop a flu, but you can help prevent cases, which I am sure it did
Veratect detected it and then disseminated their information via Twitter... the account went viral within hours
Well now that's something useful about social media for sure. Dr. Brilliant is brilliant but there is an unsung hero behind that Canadian GPHIN that really put the idea in place.
thanks for the post. It's always nice to hear the merits of social media.
Isn't Bing trying to integrate social media with search by trailblazing the "live search" thing?
They are indeed. It seems like a good idea as it is an essential move capturing the real-time web.
Good luck Bing!
This throws a wrench in my hopes for having to contend with a zombie takeover-attempt during my lifetime.
Don't lose hope! I am crossing my fingers for you.... and your brain... (Who said that? ;-p)
it is a logic