Gathering rankings is one of the most annoying and time consuming tasks of an SEO consultant's work. Because of search personalization, it can be near impossible to find accurate rankings for keywords to report to clients or to use to gauge our work's effectiveness.
Michael King and I have found a way to get rankings for the keywords driving traffic to your site directly into your Analytics using Custom Variables. Not only that, but we can also get international rankings based off of the keywords and the location from where the person came. And finally, this may also be a step towards identifying the identity of the (not provided) keywords we are now seeing thanks to "privacy concerns".
How Do I Get These Rankings Into My GA?
Different people have come up with different ways to get rankings directly into GA. One tactic is to create a custom profile using directions from this post by Chris Abernethy. This strategy works fairly well, but requires a lot of setup and requires you to be able to create another profile on your account. It also requires many steps and custom filters to clean the data.
Mike and I decided that a custom variable would be a better way to go. In order to make this strategy work, you need to be able to do the following (or know someone who can):
1) Insert the Javascript custom variable that I will provide you below into your section of your website below your normal GA code;
2) Insert a parameter into your <body> tag;
3) Be able to slice/dice your data in Analytics; and
4) Use some Excel wizardry to present the data.
The Code
Here's the code that you need to use to send the data from your site into GA. Put this code in the section of your site, directly under your normal GA code:
Get the code here
Now you need to put this code into the <body> tag of your site. Yes, directly within the body tag. This code:
onLoad="rankingsPush();
Your <body> tag will end up looking something like this:
<body onLoad="rankingsPush();">
Now sit back and let the data collect!
Where Do I Find This Data?
You find the data that is collecting under your Custom Variables tab in GA. I'll show you some screenshots. Click on Visitors > Demographics > Custom Variables, like so:
Then you'll see this screen:
Click through that and make your Secondary Dimension "Keywords". Boom! You get rankings and keywords!
What Can We Do With This Data?
Now we've got the rankings of the keywords driving traffic (which is what we really care about, right?). Data is worthless without applications, so let's come up with a few. Endless possibilities exist for slicing and dicing the data.
International Rankings
One cool thing that I did was match up the keywords and their positions with the country from which the visit came. You get the average ranking of the keyword driving traffic (which you also get in Webmaster Tools), but now you can break this down by country! Like so:
We could guess at what the (not provided) keywords are
We may also be able to guess at what the keywords coming up as (not provided) are now. As you can see in the screenshot below, I know that my visits from (not provided) are ranking #1 and #2. Based off of this, I can narrow it down to only my keywords ranking #1 and #2. If I do this for the data set I am working off of, I get these keywords:
Now that I've narrowed this down, I can take the landing pages for these rankings (set secondary dimension to "Landing Page"):
And I can see the landing page for my (not provided) keywords (Traffic > Sources > Search > Organic):
So, we can guess that most of the (not provided) keywords probably have something to with my name, since the majority went to my homepage where my name ranks #1 or #2.
*note* I am sure that someone with some mad Excel skills could automate this. If someone wants to take a crack at it, feel free and I'll link to it.
*edit* I need to give a shout out to Mike Pantoliano's post from last September where he details a very similar process. He also has some great applications in that post.
What other applications can you think of? How could we make this data sing even more?
I welcome your comments below!
Great work John (and Michael)! One thing that I'll mention is that the data will probably get a little odd over time if you use the visitor level scope, as opposed to session level. What will happen with the visitor scope is that when somebody visits via Google organic the CV with the ranking will get set, just as we expect. But if they come back to the site via a referral, campaign-tagged email... anything that will change their source attribution, the ranking CV will remain the but assiociated keyword for those visits will change to something like (not set) or whatever you may have set to utm_term. It will then be a little confusing seeing rankings associated with (not set), which is not the same as (not provided).
Ahh...good call.
So for those that are concerned with this change this line:
_gaq.push(['_setCustomVar', '1', 'Rankings', urlVars["cd"], 1]);
to
_gaq.push(['_setCustomVar', '1', 'Rankings', urlVars["cd"], 2]);
and you should be all set.
-Mike
Some more nitpicking:
xhtml elements should always use lowercase, so
<body onLoad="rankingsPush();">
should be
<body onload="rankingsPush();">
Great article, though we'll stick with the user defined value option for now: creating a new profile for this option is optional, since you can add the filter to your main profile too (assuming you're not already using the UDV). Note: I'm talking about the final filter of the linked article here. In my opinion, making changes to the html code is usually more complex than creating a filter.
right on man. hats off. :)
Hi Michael,
I understand that this question should have been asked earlier. But anyways, i have added this script to one of my websites but then number of visits tracked are almost nothing as compared to my daily organic traffic.
I have tried using both 'google.co.uk' and just 'google.' in the script, no difference.
I added a small counter in the script to see if the code was running properly and it is. Following are some figures that might give you an idea of the problem.
Date: 3rd May
Custom variable showing 1 visit.
Google / Organic 1200 visits
Counter showing 900 +
Could you please help me identify the problem?
Thanks
Great tip!
I appreciate the information and effort, but I do not see how this addresses the issue that ranking reports are a flawed metric for the very reason you pointed out in the opening of your post. With the personalization of search and blended results, I'm not really sure what purpose they serve. There are a ton of KPIs within Google Analytics that will give you a much better snapshot of the overall health and effectiveness of a website. In my humble opinion, continuing to provide ranking reports to clients is doing them a disservice. I guess my question is, whose "rankings" are you giving them? The one's you are seeing, the one's the CMO is seeing at his desk or the one's his assisant is seeing at his/her desk?
This is some really great work John & Mike, thanks for sharing. It's really amazing how some nimble JavaScript can capture a wealth of data.
@Ryan Jones
You raise an interesting point; Ranking is a fluid construct thanks to personalization, but I believe that's what makes this data all the more interesting; it's pulling the position that the particular user entered in on. It can be used to construct an average ranking profile for a keyword, or to generate a location based keyword visibility matrix to determine what geographic areas afford you better visibility.
Pairing this with a geotargeted PPC strategy to bolster yourself in the areas where you're not visible, you can make your ad spend go further and get more bang for your buck... and that's just two top of the head applications for this kind of data. It's in the same vein a Mike King's Keyword Level Demographics; deep-level metadata about your audience that becomes highly useful when a large enough threshhold is reached that you can begin to model personas or other clusters off of it. These little proofs-of-concept can build up towards something much bigger, and much better; but that all depends on your measurement plan and campaign goals. These are "forward thinking" technologies that don't offer much immediate value, but will in time ripen to precious fruits.
All analytics voodoo aside, Rank is still a very visceral KPI for clients. There is something supremely satisfying about seeing your site rank... and rankings lead to visibility, which lead to increases in traffic.
Granted it's not really a valid measure of ROI(unless the client is primarily focusing on the Brand Lift from Search Visibility), but it does point towards progress. Rank is just one of those things where client education is crucial.
Anywho, thanks again for sharing!
Some fair and vaild points made there. Thanks for the reply and valuable feedback. I always appreciate and learn from differing opinions on the subject. Have a great day!
Ryan
A small addition for international readers: replace google.com in the javascript code with your regional google version (i.e. google.cu.uk, google.de etc.), otherwise it will not work.
Great guide btw!
How can I target multiples TLDs? Should I use an 'else if' statement or set multiple instances of the script with the modified google domain?
In order to fix the TLD issue, try replacing:
// confirm they came from G if (url.indexOf ("google.com") !=-1)
{
with:
// confirm the came from G
var referrer = new RegExp("(google)\.([A-z]{2,3}/)|(google)\.([A-z]{2,3})\.([A-z]{2}/)");
if( referrer.test( url ) )
{
Thanks Kayheunen!
Haha my bad for being a short-sighted American.
Great fix Kayheunen!
Although you could just take this line:
if (url.indexOf ("google.com") !=-1)
and change it to
if (url.indexOf ("google.") !=-1)
That should work aswell, however subdomains named google would than also be accounted to the same variable.
However, I love the article.
Am I right in thinking that the rank pulling code will only work for Google.com searches and that it would need to be changed for .co.uk, .fr etc. searches?
Exactly what I was thinking. Anyone have an answer? My guess would be that we'd need to change the code to .co.uk etc. Would that be correct?
Wow!
I've now decided I'd give my right arm to be a fly on the wall when you two guys are having a quiet beer!
Thanks most of all for the clear and simple explanation.
Looking forward to the next Doherty/King collaboration :)
Keep 'em coming.
Sha
Great Tutorial! The only problem I see is that onload is unreliable and so 1997. JQuery's document ready function works perfect for this! Just load jquery from google's codebase at https://code.google.com/apis/libraries/ and you're good to go!
$(document).ready(function() { // put all your jQuery goodness in here. });
Beat me to it! Another reason to do this is when you are already using the onload event for something else. Or if you like to avoid obtrusive javascript markup in your html. Since you beat me to the suggestion of jQuery .ready I will expand on it a bit further for those who are unfamiliar with jQuery or javascript in general.
If you don't already have jQuery available, put this in the <head> of your page(s):
<script src="https://code.jquery.com/jquery-latest.js"></script>
Then put this script in the <head> right below that:
<script>
$(document).ready(function () {
var url = String(document.referrer); // confirm they came from G if (url.indexOf ("google.com") !=-1) { var urlVars = {}; var parts = url.replace(/[?&]+([^=&]+)=([^&]*)/gi, function(m,key,value) { urlVars[key] = value; }); // Push to GA Custom Variables _gaq.push(['_setCustomVar', '1', 'Rankings', urlVars["cd"], 1]); } });
</script>
To learn more about jQuery .ready go here: https://api.jquery.com/ready/
Disclaimer: I haven't tested the actual rankings-push.js. Assuming that code is valid then the above should work fine without using the body onload event.
Interesting process, just curious Does ranking data is close to Google Webmaster Tools query data?
Interesting question. Anyone has an answer?
Great question. Let me do some comparisons and I'll get back to you.
This is certainly an interesting technique, and I'll likely use it as a sanity check.
However, what's wrong with gaining ranking data from one of the automated ranking checking services? Admittedly, we have our own in-house one rather than relying on somebody elses but it never differs from a manual check.
Remember that rankings gathered using this method may be polluted by personal search.
Hey -
Thanks for the comment! I don't think thinking about "personal search" as polluting is really helpful. I think, as I said in the post, that we need to start thinking about relative, or average, ranking as opposed to absolute simply because absolute doesn't exist anymore with the advent of personalized search. When changes like this come along, I find it better to adapt, not resist.
There is nothing wrong, per se, with getting ranking data from one of the automated rank checking services. I just like having the rankings directly in my GA so I can make easier and quicker decisions with my data.
John
@jaholdensr - I think you're mistaking the true power of this information for analytics purposes.
A method like this shouldn't be deployed JUST to get the SERP Ranking of your keywords; especially since the keyword won't always be passed thanks to changes in Google's behavior for logged in users.
The true beauty of collecting rank data in this way is that it captures the position (and in ideal circumstances, the keyword) the user came in on. It's not personalization for your network or user profile, but rather for the user entering the website via Google.
Here, personalization is a good thing, as it enables you to compute a host of variables about your audience. I mentioned a few other applications here
For more raw visibility metrics, you should be looking into Universal Results. Rand talks about disabling Personalization on Moz every now and again, and it's a tactic deployed by many third party ranking tools.
This is huge! Thanks for the post.
To be honest, it doesn't appear to be working with me.
I watched a video a while back by Avinash on this topic. This post definitely helps better my understanding. Thanks for sharing!
Thank you! This is awesome! Is this not tracking every keyword? Why am I seeing 12 organic searches and 3 in Rankings for a particular day?
Possibly two factors in play here.
1) This will only track rankings through google, not all organic traffic (i.e. bing)
2) Google doesn't append the cd parameter to all queries. I am tracking over the past year that the parameter has been available about 60% of the time. When it's not there, you will see "(not set)" in analytics for the custom variable.
For some reason there is no data showing up on a test site. I added the code inside <script></script> tags after the GA code and changed the body tag as specified. Still not seeing data after 3 days.
good article I have played with ranking reports in GA before but how do you see the impact of new Google SSL search on this method?
thanks for sharing this. I have tried on a couple of sites, and on some, especially ones with huge traffic data are pretty different. In Custom Variables report I have only 10% or less of visits shown in Search Engine report. Anyone else experiencing similar?
My GA script is located in my footer.ascx file, which doesn't have a body tag. If I put your code underneath the GA script in my footer, and then modify the body tag in my home page, would that work also?
Great post man, I've just suggested this to a colleague of mine who questioned how we'd be able to differentiate between branded and non branded keywords with the respect to the technique you outlined for the not_supported keywords when they are ranking the same in the same regions and and both going to the same landing page?
Great article! I have 2 questions with my own site and implementing this. I have a WordPress site with the Genesis theme.
1 - With my little knowledge of html, I dont know how to add the piece of coding within the <body>. I can add anything through hooks before or after the <body>. Any idea on how to do this with the StudioPress, Genesis theme?
2 - I use the GA for WordPress plugin. All I do is insert the UA code. Anyone know how I can insert JavaScript immediately after the GA tracking code or does it matter?
Thanks!
this is somthing I long needed but couldn't find a solution; one problem in my particular setup thpugh. I am using a CMS and my body tag looks like this "<body {if $page_name}id="{$page_name|escape:'htmlall':'UTF-8'}"{/if}>"; problem is that if I add "onload="rankingsPush();" ; right after body, the site isn't loading anymore with a HTTP 500 error and if I add it after the if statement it has no effect.
Any idea on this ?
Robert
excellent post. Very clever use of JS and GA variables.
Great tips here, thanks for the great post and research that has gone into it. Tracking rankings is certainly something that takes up time and resources of a SEO so would be great to have this data to hand in GA so will be looking to implement this. Would be interested to here from people who try this out how you go and if the rankings seen reflect what you are getting from other ranking tools such as SEOMoz and so on.
How does this compare to the info given when integrating GWT into GA? Is it redundant?
Can someone help me out with this question?
In general, this lets you view rankings as a dimension rather than a metric.
GWT information likes to use about averages. This, unfortunately, blends everything together and blurs the context. You can't answer simple questions like, what's my conversion rate if I rank 4th? It does have other benefits, however, such as impression data.
You can use this to identify very specific details such as which hour of which day, where geographically, and what happened (Goals, PVs, Events) to visits for a particular keyword in a particular position.
You can use this to determine clusters of position performance and identify lift in engagement based on position. You can use this to try to see what the results page looked like for the user, to understand why they clicked on your link.
There are other parameters such as source=web in the google referral string. It might be worthwhile to capture that one and see what other values might be available for source.
I recommend inserting this type of code right into your GA tag rather than separating it out and calling it elsewhere on the page. The idea being that you can ensure that you're pushing and executing the objects in the ideal order.
You should mention the code needs to be placed within <script> tags.
Does custom variables work like event tracking? I just want to check that this won't affect the bounce rate by the call being interpretted as a user action.
Great article!
Hey Kitstead -
This is different from Event Tracking because it's not actually anything that the user is doing on your site. It's simply recording their actions before they even get to your site.
That said, I have not tested the bounce rate thing extensively but I have not seen an adverse effect at all.
Marvellous Stuff Doherty & Michael!! Thanks for working together & making our life simpler.
Well, there was a post on incorporating rankings of your organic keywords into google analytics a year before https://www.seomoz.org/blog/tracking-organic-ranking-in-google-analytics-with-custom-variables which was little bit confusing. But here you shown us some good & simple steps which we can easily do without any help from developer.
I also read the piece from Chris Abernathy & I must say that over there the process is long but overall I like that too.
Thanks Hyderali. It was my coworker Mike that wrote the post you referenced (to be totally honest I was not aware of it before this morning) and it is awesome. Mike did a phenomenal job.
Mike King and I wanted to make this very actionable. Chris' post provided a lot of inspiration but I like how you are able to manipulate the data using the Custom Variables better. It requires less data cleaning and I think you can slice the data more ways in GA using Custom Variables.
Thanks for the comment!
Question what if the refering google domain is google.co.za for South Africa?
uau.. really helpfull.
thanks..
Wow. THANK YOU to everyone who contributed to figuring this out! It's almost too good to be true. The data was populating within an hour of plugging in the code. I was even able to itentify the search terms of all three customers who made a purchase since then. I've been waiting, hoping, praying for this for a looong time!!
Where do I send the bottles of scotch?
R
Hey nice one, will need to test this out in the .com.au market to see what kind of results I recieve, looks interesting that is for sure...
Very interesting. I'm getting that same accuracy (50%) with some sites where I'm using the filter configuration you mentioned. Maybe I'll try this customvar thing. Good going!
Wow!! The idea of getting your rankings directly from the referrer is awesome! Kudos for this guys...!! :)
A really good description and assistance. I will try this immediately.
One of the important metrices we need in google analytics, using custom variable to get ranking is a clever move, it will help to analyze data much faster
John,
Excellent tips and nice work here. Very impressed.
Cheers
Corey
Thanks Corey. Couldn't have done it without Mike, but this is definitely testimony to teamwork and having your own site to test things on :-)
For me this setup is completely unique...!!! I mean ..I have never used the generation of Ranking reports by the help of GA. hence I would be waiting for more suggestions from the visitors and additionally...I just wanted to know are those ranking just showing the visits directly from the search engines i.e. organic visits or any other types of visits also?
well... I have got the importance of this process i.e. to show the rankings to the clients or for our own study.. and it's very quick process with real visitors response for our ranked keywords ...hence a good way to analyze those important keywords which actually give the visits!
I am curiously waiting for more excel applications to get more scoop of GA for this report..
well i am not happy with the keyword set of "not provided" due to Google's SSL implemention to the signed in users!! :(
But it is being said by some SEO experts and even from google officials that the SSL impact will be much lesser... hmmm...!!!
Thanks John!!!
good write up! this article was very cool, it gives a lot of SEO information. thanks a lot!
[link removed]