One of the most helpful functions of modern-day SEO software is the idea of a "keyword universe," a database of tens of millions of keywords that you can tap into and discover what your site is ranking for. Rankings data like this can be powerful, and having that kind of power at your fingertips can be intimidating. In today's Whiteboard Friday, Rand explains the concept of the "keyword universe" and shares his most useful tips to take advantage of this data in the most popular SEO tools.
Click on the whiteboard image above to open a high-resolution version in a new tab!
Video Transcription
Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we're going to chat about the Keywords by Site feature that exists now in Moz's toolset — we just launched it this week — and SEMrush and Ahrefs, who have had it for a little while, and there are some other tools out there that also do it, so places like KeyCompete and SpyFu and others.
In SEO software, there are two types of rankings data:
A) Keywords you've specifically chosen to track over time
Basically, the way you can think of this is, in SEO software, there are two kinds of keyword rankings data. There are keywords that you have specifically selected or your marketing manager or your SEO has specifically selected to track over time. So I've said I want to track X, Y and Z. I want to see how they rank in Google's results, maybe in a particular location or a particular country. I want to see the position, and I want to see the change over time. Great, that's your set that you've constructed and built and chosen.
B) A keyword "universe" that gives wide coverage of tens of millions of keywords
But then there's what's called a keyword universe, an entire universe of keywords that's maintained by a tool provider. So SEMrush has their particular database, their universe of keywords for a bunch of different languages, and Ahrefs has their keyword universe of keywords that each of those two companies have selected. Moz now has its keyword universe, a universe of, I think in our case, about 40 million keywords in English in the US that we track every two weeks, so we'll basically get rankings updates. SEMrush tracks their keywords monthly. I think Ahrefs also does monthly.
Depending on the degree of change, you might care or not care about the various updates. Usually, for keywords you've specifically chosen, it's every week. But in these cases, because it's tens of millions or hundreds of millions of keywords, they're usually tracking them weekly or monthly.
So in this universe of keywords, you might only rank for some of them. It's not ones you've specifically selected. It's ones the tool provider has said, "Hey, this is a broad representation of all the keywords that we could find that have some real search volume that people might be interested in who's ranking in Google, and we're going track this giant database." So you might see some of these your site ranks for. In this case, seven of these keywords your site ranks for, four of them your competitors rank for, and two of them both you and your competitors rank for.
Remarkable data can be extracted from a "keyword universe"
There's a bunch of cool data, very, very cool data that can be extracted from a keyword universe. Most of these tools that I mentioned do this.
Number of ranking keywords over time
So they'll show you how many keywords a given site ranks for over time. So you can see, oh, Moz.com is growing its presence in the keyword universe, or it's shrinking. Maybe it's ranking for fewer keywords this month than it was last month, which might be a telltale sign of something going wrong or poorly.
Degree of rankings overlap
You can see the degree of overlap between several websites' keyword rankings. So, for example, I can see here that Moz and Search Engine Land overlap here with all these keywords. In fact, in the Keywords by Site tool inside Moz and in SEMrush, you can see what those numbers look like. I think Moz actually visualizes it with a Venn diagram. Here's Distilled.net. They're a smaller website. They have less content. So it's no surprise that they overlap with both. There's some overlap with all three. I could see keywords that all three of them rank for, and I could see ones that only Distilled.net ranks for.
Estimated traffic from organic search
You can also grab estimated traffic. So you would be able to extract out — Moz does not offer this, but SEMrush does — you could see, given a keyword list and ranking positions and an estimated volume and estimated click-through rate, you could say we're going to guess, we're going to estimate that this site gets this much traffic from search. You can see lots of folks doing this and showing, "Hey, it looks this site is growing its visits from search and this site is not." SISTRIX does this in Europe really nicely, and they have some great blog posts about it.
Most prominent sites for a given set of keywords
You can also extract out the most prominent sites given a set of keywords. So if you say, "Hey, here are a thousand keywords. Tell me who shows up most in this thousand-keyword set around the world of vegetarian recipes." The tool could extract out, "Okay, here's the small segment. Here's the galaxy of vegetarian recipe keywords in our giant keyword universe, and this is the set of sites that are most prominent in that particular vertical, in that little galaxy."
Recommended applications for SEOs and marketers
So some recommended applications, things that I think every SEO should probably be doing with this data. There are many, many more. I'm sure we can talk about them in the comments.
1. Identify important keywords by seeing what you rank for in the keyword universe
First and foremost, identify keywords that you probably should be tracking, that should be part of your reporting. It will make you look good, and it will also help you keep tabs on important keywords where if you lost rankings for them, you might cost yourself a lot of traffic.
Monthly granularity might not be good enough. You might want to say, "Hey, no, I want to track these keywords every week. I want to get reporting on them. I want to see which page is ranking. I want to see how I rank by geo. So I'm going to include them in my specific rank tracking features." You can do that in the Moz Keywords by Site, you'd go to Keyword Explorer, you'd select the root domain instead of the keyword, and you'd plug in your website, which maybe is Indie Hackers, a site that I've been reading a lot of lately and I like a lot.
You could see, "Oh, cool. I'm not tracking stock trading bot or ark servers, but those actually get some nice traffic. In this case, I'm ranking number 12. That's real close to page one. If I put in a little more effort on my ark servers page, maybe I could be on page one and I could be getting some of that sweet traffic, 4,000 to 6,000 searches a month. That's really significant." So great way to find additional keywords you should be adding to your tracking.
2. Discover potential keywords targets that your competitors rank for (but you don't)
Second, you can discover some new potential keyword targets when you're doing keyword research based on the queries your competition ranks for that you don't. So, in this case, I might plug in "First Round." First Round Capital has a great content play that they've been doing for many years. Indie Hackers might say, "Gosh, there's a lot of stuff that startups and tech founders are interested in that First Round writes about. Let me see what keywords they're ranking for that I'm not ranking for."
So you plug in those two to Moz's tool or other tools. You could see, "Aha, I'm right. Look at that. They're ranking for about 4,500 more keywords than I am." Then I could go get that full list, and I could sort it by volume and by difficulty. Then I could choose, okay, these keywords all look good, check, check, check. Add them to my list in Keyword Explorer or Excel or Google Docs if you're using those and go to work.
3. Explore keywords sets from large, content-focused media sites with similar audiences
Then the third one is you can explore keyword sets. I'm going to urge you to. I don't think this is something that many people do, but I think that it really should be, which is to look outside of your little galaxy of yourself and your competitors, direct competitors, to large content players that serve your audience.
So in this case, I might say, "Gosh, I'm Indie Hackers. I'm really competing maybe more directly with First Round. But you know what? HBR, Harvard Business Review, writes about a lot of stuff that my audience reads. I see people on Twitter that are in my audience share it a lot. I see people in our forums discussing it and linking out to their articles. Let me go see what they are doing in the content world."
In fact, when you look at the Venn diagram, which I just did in the Keywords by Site tool, I can see, "Oh my god, look there's almost no overlap, and there's this huge opportunity." So I might take HBR and I might click to see all their keywords and then start looking through and sort, again, probably by volume and maybe with a difficulty filter and say, "Which ones do I think I could create content around? Which ones do they have really old content that they haven't updated since 2010 or 2011?" Those types of content opportunities can be a golden chance for you to find an audience that is likely to be the right types of customers for your business. That's a pretty exciting thing.
So, in addition to these, there's a ton of other uses. I'm sure over the next few months we'll be talking more about them here on Whiteboard Friday and here on the Moz blog. But for now, I would love to hear your uses for tools like SEMrush and the Ahrefs keyword universe feature and Moz's keyword universe feature, which is called Keywords by Site. Hopefully, we'll see you again next week for another edition of Whiteboard Friday. Take care.
Rand, Thank you for another great WBF. I love the new Keyword by Site feature. Because it uses clickstream data, I'm already finding it valuable to compare SEMrush results and overlay the two for the same domain. Gives a more complete picture. I can't wait to see where this goes in regards to local SEO. Adding a local search filter to keyword research will be amazing (I definitely use that in Moz Pro).
Totally agree on the local side -- big challenge is just tracking 40 million keywords in millions of potential locations/regions :-) We certainly hope to get there someday though.
Very important to pay attention to both types of rankings data as you mentioned Rand. Great article and it is especially helpful with the awesome update to the Moz Keyword Explorer earlier this month. For businesses that do consistent blogging, content marketing, or inbound marketing, keeping on eye on the keywords you are ranking for (but haven't chosen to track) can be a great way to discover new ranking opportunities and long-tail keywords to go after.
That is one the tools that I have been waiting for on MOZ. I'm a regular user of both Semrush and Ahrefs. Both are good with a different approach.
And is wonderful that MOZ has that new tool, will start to test it to see what happen.
The whiteboard friday is a homerun.
Awesome! Thanks Rand!
Where can I find more information about the new "Keywords by site" -feature that Moz now has?
Here you go. Rand's post from earlier this week.
Also there is a webinar that will cover features and use cases: https://try.moz.com/moz-pro-new-feature-launch/
Thanks Rand for another great post.
Hi Rand, great article, thanks!
There are so much great tools out there with a large amount of keyworddata. But it's hard work to filter out the right Keyword to optimize for. Your post shows what is all to consider to get a good list of potential keywords.
We've published yesterday a post about a experiment to predict keywords which could rank in the google top 10 via machine learning which has partial the same ideas / data you've mentioned in this post.
Perhaps we could also can set up a little experiment with MOZ Data and enhance it with some thoughts of this post.
Would be great to discuss :)
Michael
This tool's a goldmine for bloggers,
I checked it out and it gave me like 10 results which is great, though in the future I'd like to get more results. So my question is which plan do I need to buy to be able to get a lot of results ? I saw the pricing thingy but didn't really understand them, so like how much searches could I get for each plan. Thanks :D
We totally agree, this will be a very useful tool for bloggers!
Sorry about the pricing page confusion, we are in the process of getting that updated. 10 results is what logged out users get. If you are only getting 10 but are not logged out I would recommenced reaching out to our help team.
Our Standard plan allows for 150 Keyword Explorer queries per month, with 10,000 results. The Medium plan allows for 5,000 queries per month and 30,000 results. Large is 15,000 queries and 50,000 results. Finally, our Premium plan allows for 30,000 queries and 100,000 results.
Rand,
The new Keywords by site tool is very impressive, congrats!
Until now, usually we would use Ahrefs and SimilarWeb to find keywords other sites rank for, but I was very surprised by the amount of data this tool now has, and it's now part of our toolkit.
I can't even count how many different quote pages, tools and articles were developed and written on Investing.com, after first seeing them as potential keywords that our competitors rank for.
This data wasn't available until quite recently and has so much potential. Those who will ignore will simply stay behind.
Once you have the potential keywords, their volume and real difficulty (I'm not just referring to the single metric, but an entire research), you just need to convince the right people to go ahead and develop what's needed.
These are maybe SEO tools, but often the real "end users" are the Product and Content teams.
Hi Rand,
I especially like your suggestion about exploring KW sets of bigger content-focused media sites. I am definitely going to check that one out.
I don't want to be grumpy but the new Keyword research tool (by domain) that I would love to use in my daily work is not really working for sites (smaller and medium ones) in Europe (Switzerland) until now. Either it gives back nonsense (keywords that have nothing to do with the site; sites are very well in-page optimized) or no data at all. Is this going to change in the future (...honestly) or its simply due to the fact that MOZ focuses primarily on US & Canada, and maybe huge sites in Europe?
Hi Cesare,
Your grumpiness is understandable. You are correct that at the moment, our corpus is only U.S.-focused. We plan to extend to include UK, Canada, and Australia next. Unfortunately, we don't yet have plans to extend beyond that, but we're always listening for feedback and prioritizing feature updates and data expansion.
Hi Adam,
Is there a future plan to work on Arabic language, I think its going to be interesting for MOZ since Arabic Content is expanding very much nowadays comparing to the few last years.
I am using Ahrefs keyword explorer since it provides more "suitable" outcome for Arabic content, but its obvious for any SEO who practice the craft in Arabic that all tools out there lack to "precision" for Arabic web.
Notice that nowadays Arab audience (specially in Gulf region) conducts a highly monetized traffic.
Hi Adam,
So if I understand correctly, you will keep focusing on the English language. Any chance that you'll include French with your work on the Canadian market? ;)
Hi Rand, exciting topic, as per usual! I use SEMrush for keyword research, position tracking, on page SEO check and brand monitoring and I love it! Also, I'm researching quite often competitors. The great thing is that I can use it for doing research in many European countries (and what Moz Keyword research tool lacks).
Rand,
Thanks. I used the new tool yesterday for the first time. At first I was seeing what I expected. However, as I changed the filters and sorted by traffic I started to see opportunities that I have not thought of before. I have been operating the same website for 18 years, so this was exciting.
Today's Whiteboard Friday generated similar thoughts. We have several competitors in our niche, but for the past few years larger national companies have become bigger players within that niche. It will be interesting to see what opportunities we find by expanding the search to our flanks.
I have used semrush to find alternative ways to group products we already have at our ecommerce and then create new category pages.
Hi Rand
The truth is that the keywords only the extriago of SemRush and seldom of Ahrefs, but we will have to put these tips into practice
Great new feature and great last recommendation, I'll definitely give that a try.
Just one question: why would indiehackers.com want to rank for Ark servers? Are they also addicted to taming dinosaurs?
Nevermind, it's just a geeky joke. Thank you for the article!
Hi Rand,
great post. A question that I have is looking at the new Keyword Explorer feature when I search on our root domain we rank for 11 keywords yet looking at my Moz dashboard we rank for 414 keywords.
Any idea why there is a discrepancy?
Hi Atlaswealth - in your Moz campaigns, you can choose any keywords you'd like to track (and we keep those private -- we don't share them with the broader KW Universe corpus for KW Explorer). So while there may be some KWs that overlap with KW Explorer's universe, it's likely that most of the keywords you've chosen to specifically rank track aren't included there, and are unique to your own tracking, which is why the disparate numbers appear.
Thanks Rand.
That makes perfect sense. We are financial advisers to Australian expats so I guess our keywords are very niche. Great post today on link building by the way. Mental illness affects us all.
Cheers,
Brett
I'm noticing straightaway that keywords I know I rank for aren't appearing when I try keyword by site. And I can confirm by searching the keyword I feel is missing in keyword explorer as a keyword and see my domain popup as a top ranking domain in SERPs. Any suggestions as to how to be more confident with keyword by site results? :/
Hi Expio - that's as it should be! We can't possibly and would never try to track every possible keyword anyone ever searches for or every combination of words and phrases that exist. KW Explorer's universe encompasses ~40mm search terms and phrases in the US, in English. These are representative of many of the most popular, searched-for terms in Google US. But, there should be billions of keywords outside that world we don't track.
Dear Moz Team i have a very important question, i own the domain www.uae.car & www.uae.cars and whenever i use your tool to check my ranking it keeps showing error can't show data, i'm really frasturated and i wanted to use your services to track my SEO campgin with the hired company, i really don't know what is the problem exactly
Hi there, UAE.CARS! I'm really sorry to hear you've run into such frustration, thanks for letting us know. I'd recommend shooting our friendly Help Team a message at [email protected]; they will be the best people to help you find an answer to this. I hope that helps a bit!
Excellent contribution. I will certainly put it into practice.
Thanks!
Rand, this is an amazing post! Thanks.
Well put together tool - thanks for explaining it Rand :-)
For example, I use Ahrefs to search for keywords from other websites and thus find synonyms for other words that I have already repeated before. This enriches the vocabulary of that page and covers a larger number of keywords.
Good one!
Great work Moz team! I love this new tool you offer!
Hi Rand,
Great post and keyword tool!
I'm a visual person so love the Venn diagram. I like to place two competitors in there with a site I'm working on and really focus on the overlap that they have that I do not. That generally means they have something industry targeted we may be missing.
Awesome! This gave me a great idea :) Thank you