Way back in November of last year, I asked if there was some worthy soul out amongst the Internet Marketing world who would take on the monstrous task of gathering truly comparable data from the major analytics providers simultaneously, side-by-side, on several websites and publishing the findings.
Well... Ask & Ye Shall Receive.
Stone Temple Consulting's brilliant Eric Enge took up the mantle and with some help from industry brethren, has succeeded in delivering what is certainly the most comprehensive, impressive peek under the hoods of the analytics providers to date. From their interim report, released early this morning, comes:
- An analysis of how the user deletion / non acceptance rates of third party cookies and first party cookies differ.
- Comparative data showing:
- Visitors
- Unique Visitors
- Page Views
- Specific segments as defined per site for 2 sites
- An analysis of the comparative data and discussion of the following topics:
- What the numbers tell us
- Range of results
- Does one package always report lower numbers than the others?
- Does one package always report higher numbers than the others?
I'm more than a little impressed, and we should all be very thankful for this information. My big takeaways:
-
WebSideStory's HBX Analytics severely under-reports visitors, uniques and page views when compared to all the other major analytics packages (using either 1st or 3rd party cookies)
-
The third party cookie deletion / non-acceptance rate exceeds the first party cookie deletion / non-acceptance rate by a little more than 13%
-
Google, Indextools & Unica provided the most "average" counts of visitors, uniques & page views (with Clicktracks close behind)
-
The final report is going to rock.
This is the kind of data you can use to make a smart decision not only about which analytics providers to choose, but about how to interpret data when switching from one to another and how to interpret competitive data or data provided by a firm before a sale or partnership offer. It's simply invaluable.
Go read the report, then salivate with me for the final version. It feels so good not to be the only one obsessed with statistics and accuracy :)
Great, as soon as this report is up Google goes and totally updates Google Analytics
(beta release only)
Yeah, I'm not sure I like it... Its probably more usable and all I guess...
Hi Bart,
We actually did have Omniture Site Catalyst running on one of the sites in the study (advancedmd.com), and the visitor / unique visitor data looked pretty normal.
We are talking to Omniture about a more active participation, but I don't know how that will all turn out yet.
To all,
Thanks for the kind words. It helps fuel the desire to make the Final Report as good as we can make it.
Eric
Very excited for the report, thanks for initiating it Rand and for all your hard work Eric et al. It will be a nice relief to have "confirmation" that analytics are consistent or not.
It really is about time a good worthy soul (possibly nearly destroyed) did something like this. A very interesting read in deed!
I (well we) actually currently use ClickTracks for analytics of both our own and client websites. Although there are some frequent 'technical issues' I have found CT to produce fairly solid metrics and reliable reporting with all the KPIs I have yet to think up.
Very nice! Thank you.
Eric really got into the nitty-gritty on this one, didn't he? Great work. Thanks for the share.
I haven't had time to digest the report yet, but thank you Eric and thank you Rand for getting the ball rolling.
My question is, what's the point of using ClickTrends if Google Analytics is free.
The final report is fantastic, an absolute little gem. Check it out if you haven't already.
I just found your blog and read the StoneTemple report. It's incredibly detailed, and the most useful information I've found in terms of justifying discrepancies with analytics data.
I mainly depend on free analytic services like Google analytics and Cpanel awstats.
Kieth
I actually prefer Google Analytics over the paid ones.
So Clicktracks is right behind Google - that's good to know... This report is long overdue. thanks to Eric and team, and thanks Rand for bringing it to our attention. It might serve to help us educate clients on just how necessary accurate analytics tools are in ensuring the success of their online presence. Can't wait!
I've been waiting on this for a looong time... :)
Me too. *Salivating...*
I've already "ordered" my final report! :)
Good to see they included general usability as a metric.
I (and I'm sure many SEOs are in the same boat) am a bit of an analytics junkie. My favorite fix is waiting for a referal and then watching the explosion of clickthroughs in real time.
Nice post rand. Thanks Eric great good job.
Wow. That's detailed. Thanks for setting the ball in motion on that one. Thanks Eric et al for running with it. There goes another few hours ;)
Nice :)
Ah web analytics and accuracy - they're not always close friends, but more visibility into these types of comparisons helps further the cause. Thanks for helping to push the study.
Great work Eric!!
that was a large task....and kudo's to all who participated and made it happen.
While I only scanned the report and data to date.....at least its nice to know that the majority of technologies reported very similar data. If the information was widely skewed and dramatically different per methodology it would have been reason to worry.
I look forward to the time to read the prelim report and full report.
thanks for the info.
Dave
Rand,
Are they going to put in Omniture's SiteCatalyst? That page seems to blocked by my web filter. I did a small sample comparison to SiteCatalyst and Google Analytics. I compared Visits vs. Unique Visits.
As you know Visits should always be higher or at least the same amount as unique visits. Googles free analytics never reported a negative variance between the two for my client. However SiteCatalyst did.
My post on my blog can be found here if any of you are interested in checking our my tables.
Cheers,
Bart Gibby
Omniture SiteCatalyst vs. Google Analytics
This is really usefeul information Rand. Thank you
Hi Rand,
A life long question has finally been answered. I've been asking myself which stats package to use when reporting back to clients on the success of an seo campaign. Often I would find conflicting results when using Google Analytics when compared with our normal stats package. Which results do I reports back??? I hear myself ask....
I've been looking to set-up a comparison on one of my client sites using Google analytics and Clicktracks. The problem I had was finding the resource and time to complete the comaprison, and also the fact that this was of our own back, the client was not going to be charged for resource or time.
You guys have saved me a whole bunch of time..... I can now get on and give my recommendations based on the report considering it's the most comprehensive report i've read.
My compliments to all involved.
Shahid.SEO