With the recent Google Webmaster Tools security bug, I thought a deep dive into what GWT has to offer SEOs might be prudent since many SEOs may have logged in recently.
Google Webmaster Tools was once Google Webmaster Wasteland. But the past year has been a fruitful one as Webmaster Tools has rolled out improvements faster than Facebook does new privacy statements. Google Webmaster Tools (GWT) is now full of insightful data and metrics that you cannot get anywhere else. Some GWT data is useful, some is not. Let's dive in and take a look at each tool in GWT.
Guide to Google Webmaster Tools Index
| Webmaster Tools Sections | My Favorite Tools |
|---|---|
| Configuration | #1. Download Your Latest Links |
| Health | #2. View Your Website Crawl Stats |
| Traffic | #3. Submit To Index |
| Optimization | #4. Webmaster Tools Data in Google Analytics |
| Labs | #5. Rich Snippets/Structured Data Test Tool |
Webmaster Tools Home
When you first login, you'll see a list of all websites in your Google Webmaster tools account as well as few links to view all messages from Google, 'Preferences', 'Author Stats' (Labs), and a few miscellaneous links under 'Other Resources'.
All Messages
Google used to rarely communicate with Webmasters through messages. This year some probably wish they communicated a little less with the amount of "love notes" many SEOs have received. You might see a message here if:
- Google thinks your site may have been hacked
- Google detected unnatural links pointing to your site
- Google thinks links pointing to your site are using techniques outside Google’s Webmaster Guidelines
You can set the messages email threshold to: 'only important' or 'all messages' under the "Preferences" tab
See it: View All Your Messages
Labs - Author Stats
Since authorship isn't tied to a single domain, Google shows authorship stats for all sites you write for as well as individual stats. You'll need a valid author profile (go Google+!) to see stats here. The stats are interesting, and good for verifying which URLs are showing your ugly mug in the SERPs.
See it: View your Author Stats
Other Resources - Rich Snippets/Structured Data
If you've never used the rich snippets testing tool, now known as "structured data", bookmark it now. It's a one stop shop to test URLs to see if your author profile is linked correctly.
You can also use the tool to check if you've setup or verified your:
- Author Page
- Name
- Google+ Page as a Publisher
- Any structured data detected (reviews, products, song titles, etc) in the form of microdata, microformats, or RDFa
See it: Test Your URLs for Structured Data
Specific Site Dashboard in Google Webmaster Tools
Once you select a site after logging in, you see the real meat of the tool. The site specific dashboard has a nice overview showing:
- Crawl Errors
- URL Errors
- Site Errors
- Health status of DNS, Server Connectivity & Robots.txt
- Overview of # of Queries (plus clicks and impressions)
- Sitemaps (including submitted URLs and indexed URLs)
There are five major sections once you've selected a site: 'Configuration', 'Health', 'Traffic', 'Optimization', and 'Labs'. I find that the most insightful data is in the 'Heath' and 'Traffic' sections, and what you can get inside Google Analytics.
The 'Configuration' Section
Settings
Here you can target a specific country for your website, choose a preferred domain (www or non-www), and limit the crawl rate of Googlebot if you so choose.
Sitelinks
Google automatically choosing Sitelinks to display below your main URL on certain queries, usually brand related. If you have certain URLs you wouldn't want showing as Sitelinks you can "demote" them and Google won't show those demoted URLs.
URL Parameters
If you're having problems with duplicate content on your site because of variables/parameters in your URLs you can restrict Google from crawling them with this tool. Unless you're sure about what you're restricting, don't play with the settings here!
Change of Address
If you are switching your site to a whole new domain, do a 301 redirect, then make sure Google knows about it here.
Users
Ever taken like 20 minutes to add a new user to your Google Analytics account? No? OK, maybe that was just me. Luckily adding a user to GWT is much easier. There are two main user types: 'Full user' and 'Restricted User'. Restricted users are good for clients if you want to give them most view-only access, but little ability to change settings or submit things (you probably don't clients filing random reconsideration requests!).
Associates
This setting is a way for members of YouTube's Partner Program (probably not you) to link their YouTube Channel with Webmaster Tools. My guess is this section will get more settings in the future, but for now, it's very confusing. More details on the Google Webmaster Central blog here.
The 'Health' Section
Crawl Errors
Crawl errors shows you issues Googlebot had in crawling your site. This includes response codes (404s, 301s) as well as a graph of the errors over time. This is a fantastic resource for spotting broken links, as the URL shows up as a 404 error. You can see when Google first detected the error codes and download the table of errors into a spreadsheet.
Crawl Stats
Pages crawled per day is a good SEO metric to track over time. You can get some insight from the chart, but this is a metric to check in on and record every week. Ideally you want that number continuing to climb, especially if you are adding new content.
Blocked URLs Fetch as Googlebot & Submit To Index
Fetch as Googlebot will return exactly what Google's spider "sees" on the URL you submit. This is handy for spotting hacked sites as well as seeing your site the way Google does. It's a good place to start an SEO audit.
The really neat feature that's new this year is "Submit to Index". Ever made a title tag change and wished Google would update its index faster to get those changes live? 'Submit to Index' does just that. 50 times a month you can submit a page to update in near real-time in Google's index. Very handy for testing on-page changes.
Here's Matt Cutts on how to use the 'Submit to Index' tool:
Index Status
Make sure and hit the 'Advanced' button here so you can see all the interesting index stats Google shows about your site. Keep an eye on the 'Not Selected' number as that could indicate that Google is not viewing your content favorably or you have a duplicate content issue if that number is rising.
Malware
If Google has detected any malware on your site you will see more information here. Google often sends messages now if Malware is detected as well.
The 'Traffic' Section
Search Queries
These queries are when your site shows up in a search result, not just when someone clicks your site. So you may find some keyword opportunities where you are showing up but not getting clicks. I much prefer the interface in Google Analytics for this query data, and you may find a lot more queries showing up there then here.
Keep an eye on the CTR % for queries. If you have a known #1 ranking (your brand terms for example) for but an abnormally low position 1 CTR that's a sign that someone might be bidding on your brand terms (which may or may not be good). If you have a high position but low CTR it usually indicates that your meta descriptions and title tags may not be enticing enough. Can you add a verified author to the page? Or other structured data? That could help CTR rates.
Links To Your Site
This is my favorite addition to GWT this year. The link data here keeps getting updated faster and faster. When this was first launched earlier this year the delay on finding links was around three weeks. I've seen the delay down to as little as one week now.
There are two ways to download lists of links, but the "Download Latest Links" is the more useful of the two.
"Download More Sample Links" just gives a list of the same links as the latest links but in alphabetical order instead of most recent. The main report lists the domains linking to your site sorted by the number of links. Unfortunately drilling down into the domain level doesn't give really any useful insights other than the pages that are linked too (but you can't see where they are linked from on the domain). You'll find domains listed here but not in the "Latest Links" report. Bummer.
Internal Links
Pretty good report for diagnosing internal link issues. This tool is nothing fancy but URLs are sorted by most internal links. Use this to diagnose pages on your site that should be getting more internal link juice.
The 'Optimization' Section
Sitemaps
See a list of all the different types of sitemaps Google has found or that you have added and some stats about each one. You can also test a sitemap as well before submitting it and Google will scan to find any errors. Webmaster Tools shows stats here on Web sitemaps, as well as Video, News, and Image sitemaps as well
Remove URLs
You can submit URLs (only for sites you control of course) that you wish removed from Google. Make sure and follow the removal requirements process.
HTML Improvement
Think of this as a basic On-Page SEO audit tool. Google will show you lists of URLs on your site that don't have unique Title Tags, or are missing Meta Descriptions. This is a handy tool for quick On-Page SEO issues when you first take over a new website. Click on any of the issues found to return a list of the URLs that need improvement.
Content Keywords
See a list of single keywords, not key phrases, which Google thinks your site is about. As long as you don't see spam stuff here, you're good.
Structured Data
If you have some structured data on your site, such as a linked Google+ author or product review data, you can see stats about that data including the type of data found and the schema. This is useful to mass verify that all the pages you think are marked up correctly actually are.
The 'Labs' Section
Custom Search
Ever wanted to build your own search engine? You can with Google Custom Search. If you have a collection of sites that you're always searching through using Google, you might consider using Google Custom search to build your own Google that just returns results from those sites. You can see how the custom search engine would work on just your own site using the preview tool here in Webmaster Tools.
Instant Previews
Input any URL on your site (or just leave blank and click 'Compare' to see the homepage) to see what the preview of the site might look like in a Google desktop search results set, or on a mobile SERP.
Site Performance
This tool got dropped by Google's spring cleaning in April 2012. I like using webpagetest.org for testing site performance.
Webmaster Tools Data In Google Analytics
Connecting your Google Analytics account with your verified site profile in Google Webmaster tools brings some GWT data directly into your Google Analytics account. No need to login to two places.
To connect a verified GWT site to the correct analytics site, click the "manage site" dropdown:
Once connected, GWT data shows up in the Standard Reporting section of Google Analytics under "Traffic Sources" -> "Search Engine Optimization".
Not all GWT data is available in GA. You'll only get three data sets in Google Analytics:
- Queries
- Landing Pages
- Geographical Summary
Let's look at each of these and see what's worth looking at.
Queries
Queries are interesting because you can see some of the keywords that might be hidden under (not provided). This doesn't help with attribution of course, but at least we can still use that data for keyword research. Darn you (not provided).
What's really interesting is how many more queries show up in the query report in Google Analytics (that is supposed to be GWT data) than do when you directly get the query data in Google Webmaster Tools. For example, for the timeframe: Oct 28th-Nov 27th we had 317 queries report in Google Analytics:
but only 93 in the Google Webmaster Tools 'Top queries' report:
I'm not sure why such a big discrepancy between GWT queries and queries in Analytics from GWT. I definitely see more Google Images type queries in the GA report and less in the 'Top Queries' in GWT. Interesting discrepancy. Anyone else notice a big difference in query data?
Nonetheless the Query data can be interesting and it's nice to have right in GA. I hope that Google continues to provide more GWT data directly into Google Analytics like this.
Landing Pages
You're better off getting your actual top landing pages list from Analytics, but you can see what GWT sees as your tops pages sorted by Impressions. The interesting nugget of info here is the CTR. That's not data you see in analytics and could be insightful. I like comparing the CTR to the site average:
Geographical Summary
This section is again useful really for the CTR rate data. Looking at specific countries you can see where it might be worth running more Facebook ads or doing some international SEO work in.
What do you use Google Webmaster Tools For?
OK, I've ranted enough about what I like in GWT. What about you?
What data do you find useful in Google Webmaster tools?
Great post Scott!
GWT is awesome! A big part of our SEO process and a great place to find keywords that you're getting impressions for, but may not be getting a lot of traffic... yet. The structured data tool has become indispensable and the sitemap submission tool for those XML's & KML's is great as well.
For page speed I really like Google's Page Speed Tool and also use webpagetest.org, but I have found https://tools.pingdom.com/fpt/ is a great tool to add to the belt as well to provide some great insight that the others don't sometimes.
Have a great day!
Yes, absolutely agree on Google's Page Speed tool. We use it all the time in assesing the Page Speed. It gives some very helpful suggestions on what you can implement to improve.
Nice article! I find it humorous that so many people want to know how to improve their search engine rankings on Google, complain that Google won't tell them what to do, and they overlook the information rich asset that is WMT. This tool has served us well to double check our work, and to confirm with clients the particulars of essential metrics having to do with their website. WMT is a must-have in your toolbox!
I find the all the link information they've added over the last year or so to be absolutely indispensable. I do applaud them for giving as much information as the do about who's linking to us, how they are linking, what our most linked pages are, and what the most recent links are. It's important to see the links that Google sees because a lot of the main backlink analysis tools won't show you those nasty little baby links that can be hurting your site.
I didn't know they dropped the site performance. Why do you think this is? I always thought it was a bit inaccurate, calling any site slower than 2 seconds a slow site.
What do you think Google should provide us now that it's not currently?
I agree with you in a bit, but for the most part i think that Google WMT is a great tool in hand and no other comes close to this. I don't see any replacement for it or anything new, unless its a great updation. Pretty healthy research and info, by the way.
Outstanding post Scott, did you write it before Data Highlighter was launched or did you leave it out as an advanced feature?
Yep, that came out between when I submitted the post and when it got published!
Anybody know what the latency is for displaying the actual info via "Search Queries" tab? Thanks!
Well structured and informational post. Thanks for sharing your experience with us!)
Webmaster tool improved to a greater extent over the last few months... and 2012 overall was a great year for Google Webmaster Tool. As a roundup post on Google Webmaster Tool it was a good read but i still think GWT have a long way to go...
Web Master tools is way underestimated as a tool, as a primary tool - unfortunately. Great guide - hope it will stick with some SEOs that are not currently using it...
Wow, this is such a comprehensive guide. Thank you. I'm bookmarking and off to check my webmaster tools again. I've used the broken link section many times and found that helpful. I've been making a bunch of tweaks to many pages on my site over the last 3 weeks or so, so it will be interesting to see the impact. I also like looking at the CTR. Shows me which are doing well as well as some that have room for improvement.
This is really a stunning post and I am really appreciate with that kind of posts. Google Webmaster playing a very important role in today's world. I want to know about Disvow tool which is recently launched by Google. I want to add it in Google webmaster tool(GWT). Give a rough idea about that remaining I will do myself. Because I think the person who doing their work by himself and doing hardwork, he will be successful in future.
Thanks for sharing this fruitful information.
GWT is one of my favorite tools. Google has made very good changes in 2012 at a very fast pace. Fetch as Google BOT, Website inbound links, Sitemaps, & Index status sections are interesting parts of it. Thanks author for such a great post on GWT. I've seen it on mozblog after a long time. Its a great guide for beginners and would definitely help them in optimization.
nice article and a good read.
I got an edit for you: Submit to index is actually available 50 times per week (Matt Cutt's his words) and you can request a full audit (crawl of the page + all pages it links to) 10 times a month.
Really nice feature.
Again, nice article.
Regards
Jarno
Great resource. I have already bookmarked it to keep recalling. I recently joined a SMO company and thus, new to SEO and not that expert in analytics. This post helped me greatly. Thanks for sharing.
Outstanding post. Really very useful.
Very Informative Points.Google Webmaster tools is one of the most important tool available to webmasters, also help to monitor website functionality and search engine optimization results.
Webmaster tools is a great tool, my only gripe is the time it takes to update the info.
Google Webmaster tools is one of my favorite tools as an SEO. This guide is great for people to get acquainted with this powerful tool. This is a great cornerstone guide for all SEOs.
Does anyone have any insight about how you can find out more info about the pages that end up in the 'not selected' index?
WMT is definitely a must add for any webmaster. It will help every webmaster know what is going on with their site in relation to Google and their algorithm.
I personally visit every 1-2 days to check messages and errors on the site.
Awesome post, great to get an overview of the most recent features.
Have you tried using the 'demoted sitelinks' function much? I've not had much success with it and the help files say it's not guaranteed that they'll drop the links you demote (https://support.google.com/webmasters/bin/answer.py?hl=en&answer=47334).
Thanks for the updated guide.
As for the most useful data, I've always found it indispensable to have the warning messages about problems. If you have all your clients on WM Tools, then you've got an automatic website auditing system to alert you by email.
Thanks for the write up on GWT. We recently added rich snippets to our e-commerce site in November but have not seen the data show up in the SERPs. How long does it typically take for the data to show within SERPs?
This is so nice guide for every user.
Any body can tell me whats makes real difference between Google webmaster tool and Moz pro Tool. I think Google webmaster and Google analytic showing enough data analysis without any cost. So why should we spend $99 per month on MOz. ?
Thanks
Mazhar
We have a Getting Started guide at https://moz.com/help/guides/getting-started that can give you an overview of Moz tools and help explain what we do.
Hi everyone,
It's just recent, but it seems that I can't download any of my latest links. Has anyone encountered this, and how did you fix it?
Hey Guys i need your help to submit site on google webmaster tools...to Verify ownership of my site am using HTML file upload method i have uploaded file to my site see
https://999hdwallpaper.com/google8f19a919a1db94e7.html
but google webmaster showing..
We weren't able to verify your site: https://999hdwallpaper.com/...
Please Help me guys......
My site
https://999hdwallpaper.com
Have you tried asking in Google's Webmaster Help forums?
Hi there. What's the simplest way to check if my site was verified correctly in Google WMT; i think it was but want to make sure.
What about SEO tool kit of Microsoft. I think its just perfect and free of cost :)
in my web master tools after disable google analytic for 3 day and enable it now 3 days behind
When I use "Site:yoursite.com" to look at the number of pages that are indexed, the number that comes up doesn't match with the GWT Index status data, Why is this happening? Which is more accurate information?
Amazing Article that covers all steps to know "how can we play with GWT".
i have a problem with my site https://opakistanidramas.com/ that google is not indexing my pages.. my site is 7 months old. i think the webmaster settings is not good. do you help me to make good setting for indexing. plz
Awesome post by beammeup. I got the all information here.
just what i needed Thanks for updating !! and keeping it relevant :)
I Don't Mess With Other Options Of Google Webmaster Tools,Have Tried Fetch As Google Bot Only,Because I Am A Beginner To These Kind Of Stuffs.But After Reading Your Post,I Will Definitely Jump Into Other Options As Well.
Thanks For A Helpful Informative Post!!!
[link drop removed]
Question for the community: Are the results in the query section of GWT geo-centric? In other words, if it shows me that I've shown up for "Buy Widgets Online" does that include queries that contain "Buy [Location] Widgets Online" or "Buy Widgets Online [Location]"?
Furthermore, should we assume that some of these queries, although generic or broad, generated a listing in the SERP because the search was preformed from an IP or user close to the designated website location, if geo-centric SEO is incorporated (e.g. kml, NAP, geodata)?
I am writing this to get some help from Google webmaster tools expert, since two months I have submitted my website in Google webmaster tools and in "Crawl Stats" section I am seeing the following data.
Pages crawled per day - Missing graph data for some specific date
And the same is the case with "Kilobytes downloaded per day" and "Time spent downloading a page (in milliseconds)".
I don't know what's wrong with my website and whether this Is any server problem or something else.
I would like to have expert suggestions for this problem.
Looking forward to positive reply from you.
Regards,
Azeem Tamboli
Hey Bravo!!Finally got the brief guide on webmasters. It really gonna help me to make our trainees be aware of webmasters easily and closely who used to digg me with so many queries.
Is there any such guide for Google Analytic? Anyone? :)
I am so glad I stumbled upon this post! I've always been frustrated with the tools not knowing what they're for or how to use them properly. I like most people don't have time to sit down with a 500 page user guide trying to figure it all out. Thanks for the concise guide!
I loved the article few days back i have started using the webmasters for my reporting purposes and i was unclear about some points of lab section. but here i got my ambiguities cleared. thank you
Disavow link tool is my favourite and i use it on all low DA sites that are junk on a regular basis.
Why does SEOMoz toolbar tell me i have 3 times as many links as GWMT?
And why has my PR gone from 3 to 5 when my queries have gone from 17000 to 700?
"Why does SEOMoz toolbar tell me i have 3 times as many links as GWMT?"
No doubt owing to the fact the Google has avowed to never disclose all links that it knows of.
"And why has my PR gone from 3 to 5 when my queries have gone from 17000 to 700?"
Because PageRank is independent of traffic.
Google Webmasters Tools omit 'fluffy' irrelevant links which neither help nor harm your site. Also note the difference between links and linking domains, not sure if you've mixed the two up between GWT and MozBar.
Useful post - Even Google's own guides for WMT shows out of date screenshots!
You mentioned "why such a big discrepancy between GWT queries and queries in Analytics from GWT".
As far as I know, this is because google takes your Adwords positions and your organic keywords positions, and then averages them out.
This would not explain the reported difference in the number of queries reported.
Perhaps WMT reports organic data only, while GA/Urchin reports organic & AdWords.
Great article beammeup- some great insights there!
I've also noticed a discrepancy in the queries count between GA and WMT but the reason is that WMT filters the list to 'Web' by default, whereas GA shows the unfiltered list.
Either click Filters in WMT and select All from the Search drop down box, or add 'Google Property' as Secondary dimension in GA, then add a filter to 'Include Google Property Exactly matching Web'.
The total number of Queries between GA & WMT will now match.
This is certainly the case where there is no AdWords traffic. Maybe someone else can check what happens when there is mixed traffic.
This reference on Google I found useful too:
https://support.google.com/analytics/answer/1308626
Pretty thorough list... The not found list has gotten worse and worse
Loved this post. Very helpful guide I must say. This is my new go to for webmaster tools.
One thing that is not on Google webmaster tools but IMO it should be is Google's Page Speed Tool. Why they keep these separate is beyond me :)
Timely ... going through a steep learning curve with Webmaster Tools, so this has been most informative. Fetch as Google Bot is actually quite a nice featureSteelasophical
Great post about GWT. But is their any method to check website is verify in google webmaster tool, without login in GWT?
I think WebMaster Tools is very under appreciated. We spend so much time in Google Analytics that we often forget the importance of Google WebMaster Tools.
Thanks for the post, it was greatly appreciative!
Thanks for the post - I use WMT pretty much every day but this post inspired me to go get some downloaded data and setup a schedule of downloads so after this month I have all my data going forward including the WMT AND analytics stuff (not 3 months worth.)
The URL parameters tool is extremely useful. Our clients' ecommerce sites typically have lots of pagination and search functions.
Very nice post. I didnt went to google webmaster lately, but I will review it all again now :)
Thanks you!
Thanks for the post, Scott.Quick question -
Why does the 'Index Status' page show a different number than typing "site:www.example.com" in Google? Which is more accurate?
I would trust the number from Google Webmaster tools as the more accurate one. At least more consistant. Google operates each division in silos and often data between the silos varies, so I'd stick with one and report on that over time.
Crawl stats page in Google webmaster tools has been updated couple days ago.
Awesome sauce ... thanks! :)
Maybe its just our site, but it seems google updates its links in GWT every 4 days, not 10 ?
I like GWT since using it. But most interesting things for me is "Search Queries", "Link to your site" and "Index Status" (new feature). But now I would suggest Google Developer to have all indexed URLs should be listed under "Index Status". So that webmaster can download and save time in analyzing indexed URLs.
This is a great post, thanks. I love the tip about comparing CTRs in Google Analytics. Great reminder, too, about using the Submit to Index tool.
Thanks. Great recap and update on GWT. :)
Google webmaster tool is really nice one to use for all the SEO purpose, but i tried to create a robot.txt file with this tool, couldn't able to create it, i don't know why but it is really a good tool for the remaining works in SEO. Excellent blog and its looking informative. Thanks for sharing.
Gr8 Post....
....mentioned everything anyone need to know about webmaster tool...
Pretty handy share you provided here. I'm new to this things so This is a big help for me.
Thanks for sharing beammeup!
A German guy wrote a even more detailed ebook about Google Webmaster Tools earlier this year. You can find it here: https://www.trustagents.de/unternehmen/publikationen/google-webmaster-tools-e-book
Cheers
Thank you, Scott, for pointing out that the GWT queries can be viewed in Google Analytics. Your guide is great for exploring GWT, and much appreciated.
Thanks! Best of luck exploring!
well done for amazing post Scott you described all the internal features of GWT all those are very important points like: impressions,html suggestions, sitemap, site links etc.
Great list Scott - will definitely be reviewing all these features in full in 2013. Particularly interested in the Author Stats, very cool indeed.
woww.. awesome describe, along with I like Disavow feature. But is there any option to down load all my back links with landing pages (not domain) through webmaster tool? because I have 1000++ back links and it's impossible to collect all links manually for Disavow.
Great article, anyways thanks for the info, i didn't notice this.
Nice post Scott. I'm trying both Google Webmasters and SeoMoz - both seem to similar in helping me determine my keyword rankings. However, does Google also pick up on geographic location and IP for search results or does SeoMoz follow suit? For example, when I Google "how to get more followers on twitter" my site always shows up on page one on my laptop. However, when I do the same from my parents' place or the airport, my site hardly shows up for anything. Do you think it's being targeted geographically or is this something totally different? The traffic results from Google Webmaster Tools and SeoMoz seriously differ from the search results I'm getting on my home laptop. Hope this isn't a stupid n00b question but which one do I trust?
Your site appears always in the first page on your laptop because of personalized search, not because of geolocalization...
Hi
As you said when you search "how to get more followers on twitter" on your laptop then you see the top results for your website on first page. But on another places you can't see the same. This is happening just because of the cache of your system and your browser. Next time when you will do search this keyword in your laptop then first clear cache of your browser (ctrl +shift + del) and then restart your browser. Do research of this keyword again and then see the results.
I don't see submit to index under Health category? Can anyone tell if it is just live for US webmasters and Google hasn't rolled it out internationally?
"Submit to index" option show-up after fetching a page. "Fetch as Google" can be seen under Health category.