Taking advantage of and capturing searches within the long tail has become an important area of focus for search marketers. It certainly has tremendous appeal… typically long tail searches have considerably less if any competition yet often represent searchers who are very far along the purchase path. Seems like the best of both worlds.
The challenge of course is in actually identifying these searches. To some extent, just having pages with good, high quality content, in this case content that contains key features, benefits or other details that searchers might search on, may capture some of this traffic with little extra effort. But what if you want to take a more proactive approach to the long tail?
Hamlet’s article about Uncovering the Invisible Tail demonstrates the challenge. Based on keyword research tools alone, much of the long tail may not be found, in essence these terms are invisible, flying under the radar of most research tools, which to some extent may only capture the head and mid-tail.
There are two big challenges that are inherent in long tail searches. Long tail searches are like comets that only come around every so many years. Identifying them today may not necessarily deliver value until some unknown time in the future… assuming we live long enough to see them again. Secondarily, their overall value is found in total, not in individual searches.
So how do you increase your chances of coming up for these searches in greater frequency while minimizing the effort expended? Through identifying long tail patterns, of course. Like our comet example, we are less interested in the trail, but in the path the comet is taking. In other words, we are looking to identify where the comet will be by mapping out where it has been.
With our keywords, we do this by examining the searches that we have found as far down the tail as we can, either through the keyword research tools, using a service like Hittail or using a strategy like Hamlet detailed. The whole goal is to identify key patterns that we can model our efforts after, ideally efforts that can be automated to some extent. Life is full of patterns, even in search. While we may not be able to capture every search, by identifying dominant patterns, we may dramatically increase our ability to come up for more of these searches.
As a hypothetical example using the ever popular digital camera example, here are 40 searches for two different brands and models of digital cameras that have been pulled (for this demonstration) from the Keyword Discovery database that received only 1 search:
- consumer comments on nikon 5.1 mp coolpix l3 digital camera
- new nikon coolpix p3 8 1 mp digital camera memory
- nikon 3 2 mp coolpix digital camera
- nikon 51 mp coolpix s1 digital camera and cradle
- nikon 6 mp coolpix digital camera
- nikon 7 1 mp coolpix 7900 digital camera
- nikon 81 mp coolpix 8800 digital camera
- nikon coolpix 4800 4 mp digital camera
- nikon coolpix 5200 51 mp digital camera
- nikon coolpix 5400 51 mp digital camera
- nikon coolpix 6.0 mp digital camera
- nikon coolpix 8700 8mp 8x zoom digital camera 8 mp
- nikon coolpix l2 6.0 mp digital camera
- nikon coolpix l3 6 mp digital camera usa warranty
- nikon coolpix p2 51 mp digital camera
- best buy sony cybershot dsc t7 51 mp digital camera
- brand new sony cybershot dsc h1 51 mp digital camera
- camera digital sony cybershot 51 mp
- sony - cybershot 10.1 mp digital camera
- sony - cybershot 6.0 mp digital camera
- sony 5 mp cybershot dsc t9 digital camera
- sony 72 mp cybershot dsc p200 digital camera information
- sony 72 mp cybershot dsc w7 digital camera
- sony 72 mp digital still camera cybershot rebate
- sony cybershot 10.1 mp digital camera
- sony cybershot 7 2mp digital camera 7 2 mp
- sony cybershot 72mp dsc w7 digital camera 72 mp
- sony cybershot 81 mp digital camera
- sony cybershot digital camera 5.1 mp
- sony cybershot digital camera 6 mp
- sony cybershot dsc 1 81 mp digital camera review
- sony cybershot dsc h1 51 mp digital camera
- sony cybershot dsc w30 6 mp digital camera
- sony cybershot dscs40 41 mp digital camera 3x opt zoom
- sony dsc p73 cybershot digital camera 41 mp p 73
- sony dsc p8 cybershot 32 mp digital camera
- sony dsc s60 cybershot digital camera 4 1 mp
- sony dsc s85 cybershot 41 mp digital still camera
- sony dsc t1 cybershot digital camera 5 0 mp
- sony dsc t1 cybershot digital camera 50 mp t 1
So our goal is to determine if there are any universal patterns that searchers may tend to use when searching. Within this subset of searches, there are a number of patterns that stand out:
- ~48% begin with the brand name and end with “digital camera”
- ~35% are ordered brand, model name, model number, megapixel, “digital camera”
- ~22.5% are ordered brand, megapixel, model name, “digital camera”
- And a whopping 60% follow the overall pattern of brand, model name, “digital camera”
You might also notice that, at least in this example, that qualifiers such as new, a specific store name, and a reference to consumer comments tend to be pre-pended to the search phrases whereas features and product related qualifiers tend to be appended to the search phrases, such as memory, 3x opt zoom, warranty, cradle, information and even a repeat of the megapixels or model number.
Remember, this is purely a limited, hypothetical example and certainly not meant to be statistically exact. Actual analysis might reveal different patterns and this was meant merely to demonstrate the idea of identifying dominant long tail patterns to model content presentation on, whether for static use or in pulling data from a database.
identity,
Thanks for referencing my article, for making those valuable comments, and for bringing a wider audience to this interesting challenge.
You make an brilliant point about identifying patterns in order to succesfuly capture invisible long tail searches.
Thinking about technical approaches to automate this will require some serious research, but my basic idea is this:
1. Create labels for each entity. ie.: brand name, generic name (digital camera), model name, model number, attribute (megapixel, etc.). Each label represents a set of possible values.
2. Generate all statistically meaningful permutations (different combinations) of the different labels.
3. Generate probability weights for each permutation. In your example: 22.5% probability for the brand name, attribute, model name and generic name permutation. The probability weights would come from the historical use of such search patterns.
4. Choose the top 10-20 permutations with the highest probabilities. Make content for those.
Folks, I think you are making it overly complex, than it is.
The word order in search queries plays little role, as compared to presence in title, body tags and links. That's why it is sufficient to target 4-6 word keyphrases (sony cybershot digital camera) and create content acount around them.
While this may seem as a general phrase, if you create a whole section about this camera (and its versions), you'll essentially be capturing 80-95% of the tail you'd otherwise get by also paying attention to word order and other stuff.
What is useful in your findings is, of course, the types of words people use in long tail phrases. I think more research should help in this direction.
A.N.Onym,
Thanks. You have a valid point. Search engines are smart enough to match our documents no matter the order of the words.
Please note that I used the words "statistically meaningful permutations". Word order is one of the variables. Using the right words is another variable.
The beauty of automation is that the technical details might be complex, but using the software once it is automated, does not need to be.
I am inclined to believe people will be more likely to click on results that match their searches exactly, than those that don't. I would rather have the words in the right order, and have higher proximity scores.
Sorry, Hamlet, but content generation (not creation) automation is on the gray, not human, side of things for me. You are, of course, right in the realm of software generated content.
A.N.Onym,
I wear the same hat as you :-) I am not endorsing the use of automation for content generation. Sorry if my previous message was not very clear. I find it hard at times, to make technical ideas easy to comprehend.
All I am suggesting, is how to automate the identification of the patterns identity is writing about.
A.N.Onym,
I agree to some extent... this is more micro level and can hit diminishing returns very quickly if you aren't careful.
Absolutely, certain elements are far more critical in ranking value than pure word order, but if I can increase my chances (again talking global scale here across pages) of coming up for more searches or higher in rankings because my word order was more exact, or increase click-thru because my pattern matched the searchers pattern, then I'd want to do that.
Yes, you certainly want a page that talks about Sony Cybershot Digital Camera, but what this aims at is all those pages under that page as a way for pseudo-mass optimization.
I'm in a different kind of long tail pickle: We have a client that has an unusual technology used in medical device product development. Their long tail consists of keywords where the searcher doesn't know what they are looking for until they find it (an aha! moment). All very technical. So not only do we know there's a long tail but we have to bid on all of it because, and this is the kicker, we want to capture certain searches even if they happen once a year! That one searcher might be worth millions to my client. It's a crazy challenge but someone has to do it...Someone said they hate to do the work for one click a week- imagine one click a year. This is the future we're staring at.
> Their long tail consists of keywords where the searcher doesn't know what they are looking for until they find it (an aha! moment).
That is the very fire in which HitTail was forged... not knowing the exact words that are being searched on. Blog sufficiently about the topic. The long-page versions that are the index and archive pages will make your site get served in the search results for coincidential word combinations you never thought of. A word from a post at the beginning of the month combines in an unlikely fashion with a word from a post towards the end of the month.
Once you have 100 or so posts "seeding" your site, the suggestions start coming in.
The obscure topic on which I started playing with long tail strategies is "digital signage" for using flat panel TV devices as signs. Imagine the variety of ways people go about researching that without knowing what it's called. And these marketing techniques turned that company into the leading world provider.
Sometimes you have to find the connecting points. Obviously this is especially challenging for new or cutting edge area, technological or otherwise.
When you invent a new mousetrap and it is different and better than every other mousetrap in the world because of X, that's all fine and good unless people aren't searching for or even know about X.
So we look for connecting points... back to marketing and identifying needs, problems, and desires. Bring them in based on what they know, then show them how X is the answer... it meets their need, solves their problem, etc.
This is more challenging, but can often be more powerful.
We have developed a software application that we use internally to help us uncover and better target keywords/phrases that have the potential to appear in the long tail and thus increase this type of traffic to our client websites. In the spirit of SEOMoz, I'll share the logic behind it with the readers here.
Now comes the good stuff. Using our application, we can now choose any individual word we want and drill down recursively to determine which other set of individual words appear most often with this one. This can be done based on a number of factors including relevency, keyword demand and competition etc. Using this process we create what we call Keyword Implementation Guidelines that are typically passed off toCopywriters to use as a basis for writing long tail targeted and optimimized copy for the page; andWeb Developers to ensure proper implementation of the page's title tags and other metadata. Using the digital camera example, we would determine the set of individual words appearing most often with the words 'digital' and 'camera' (and others if need be) from the Keyword Discovery data set. We'd then write copy for the page using 'digital camera' as the target phrase while ensuring that all the words that appear most often with these two words are also placed in close proximity with the target phrase thus reinforcing the long tail theme.
This entire process is usually complete within a week or less. As you can imagine, it's very rewarding when we review our client's log files to reveal the amount of long tail traffic coming to their websites. There are always things there neither us nor the client would've thought of initially.Without this process, we could easily sit around for months trying to accumulate all the possible combinations of words that can appear together. Then, trying to target these would be a huge challenge in itself.
The pattern of Brand, Model Name, “product type” is interesting to note. I will keep that in mind for the future.
An often missed opportunity for e-commerce sites is parsing internal search logs. What people are searching for internally can be quite different than what they arrive with.
Your greatest potential for the conversion is capturing a strong ranking for an internally popular phrase. In many cases internally popular phrases are in the long tail of the search engines, and thus easy targets.
Now I'm far from knowing anything about SEO except some common sense here and there.
When I first started our site and was attempting my own keyword research, here is what I did.
I asked everybody in my family (9 in all) to assume they wanted to fix their credit. Jump on the internet and see what they can find for me.
Once they all did, I asked them what they typed in to find the info they came up with.
Amazing results. All 9 had the same instructions, all 9 had entirely different search phrases.
This is an important ongoing process. I add unique terms from organic traffic every day now. Use SEOBook's misspelling generator then Add "" and [] to each one. We have a few new ones everyday.
Good stuff.
I really like your analysis; my frustration with long-tail targeting is that I hate putting a ton of effort into something that will net 1 click/week, but I think your approach of analyzing the patterns in the tail is a great one. The other frustration I find is in doing long-tail PPC work. Google and Yahoo both have built their systems to reward high click-thru, and sometimes the long-tail terms that are perfect for my clients (just cheap enough to be profitable, and just frequent enough to be worth investing in) get penalized for being too inexpensive or having a sub-optimal impression rate. Once they get pegged as being low quality, they either get dropped or the bid-rate jumps. That can be really irritating, as it defeats the entire purpose of going after the long tail.
Doc, you hit the nail on the head... where's the breakeven point between "putting a ton of effort" into what it nets in the end.
That's where trying to identify patterns will hopefully help out... the long tail isn't really about the "1"... it's about lots and lots of ones, hopefully bringing a greater return as a whole.
So now the one that you are trying to find represents many. Thinking at it from that standpoint, we are once again looking for the 80/20... 20% of the patterns that represent 80% of the long tail.
Another problem with capturing long tail searches for low PR sites is that so many of the pages that you put effort into for some of these searches just end up in the supplemental index.
Here is another element that one may consider for capturing hits from the long tail.
As we know the on page factors can be augmented through keyword density, proximity of the keyword in relation to qualifiers and used as pivotal terms. Semantic interpretation (LSI) also has a lot to do with it and each search engine has a unique method for identifying the context of qualifiers / keywords. Yahoo and Google will see the same page differently whereas Google will respond in kind to off page link building in my opinion much faster than Yahoo or MSN.To summarize if you think of the page like a density cloud, identify your most pertinent keywords (using a keyword density tool), find the keywords with the highest occurrence and then use them in two or three word off page anchor text.
Most keyword density tools already provide the results, just mix variations when link building. Having 5 unique variations is oftentimes enough to rank competitively for any combinations of the long tail search (company, services, effective, affordable,etc).We used this method primarily to develop our own site and as the links age, we get even further traction for each respective combination as the key phrases move up the tail. Of course tracking or determining where the head goes can be tricky, but the point I am trying to drill home is, strategic link development is basis of long tail optimization.Aside from that, using exact match root phrases in the title and description tag, serve to act as signal flares for the tail and are excellent ways to also capture long tail search.Every few weeks, go back and mix up the descriptions and vary the titles to add more variations to the bin or (Google Ghost Index) that stores all of such changes. Also putting the qualifiers in block quotes is another known way to garner attention from bots in conjunction with the obvious on page seo methods.
Very Good Article :)
Brian,
You said:
"The whole goal is to identify key patterns that we can model our efforts after, ideally efforts that can be automated to some extent. "
We just released a new SEO/SEM app called Concentrate (with paid and free versions) that will automate the anaysis you describe here and automatically find patterns in the long tail.
I reference your post in a write up I did yesterday on the Juice Analytics blog: "Target Long Tail Searches with Keyword Patterns" which shows an example of how to extract long tail patterns in Travel related searches.
Hope you find the tool helpful, let me know if you have any feedback.
-Pete
When I do keyword research, I look for two words that receive considerable search engine traffic. Then I think of words that are associated with and commonmly used with the primary keyword term. This allows me to find keywords with traffic and also that if someone saw the term, it would easily be remembered because of its natural association with the main keywords. It's that simple.
Google is not in favour of showing the true converting keywords. One of the reasons why google does not give you the Indirect keywords is that it dilutes the bidding price model.
The brand keywords in my opinion keep the majority of conversions, but it is in non brander (indirect ) keywords that hold the higher weight.
I believe there should be 3 models - non branded/generic (the browsing phase ) the interest/ research phase (mixture of branded and non branding) and the impulsive phase (product or brand) .
I believe that the distribution of weight of the conversion should be through the sequence and not just on the last click conversion.
Please Help – Free Tool for feedback
I have a free tool that is designed for this one reason, this tool will identify all the keywords and tell you the average delay, the converting keywords and sequence in which they occurred. I would like to test it on the community and to gain their feedback. I have gone through the development stage but would like to give it away to set users. All help will be very much appreciated. If you would like to know more please send me an email on [email protected], and i will send you through how it works.
The tool works exactly like analytics but gives you all of the keywords/ phrases typed. The results will generally take 30 days to see although the information is live. I am in the stage of designing a bubble graphic interface but should be finished by the end of the month. This is why feedback will be very useful.
Great article - love the examples of the digital cameras
Some niches are not so product-name or brand specific when a service is targeted, so it can be very difficult to find the long tail phrases for these, as more than 2-3 keyword phrases tend not to show up in keyword suggestion tools or analytics.
I will try some of the suggestions here.
Great post, I like the concept of identifying patterns for the long tail searches as it can help us tailor more of our content without putting out too much effort.
A question about landing pages. I've been reading up alot on landing pages and feel I have a fair grasp of a well organized landing page.
My question is submissions and linking to it. Do you just submit the new landing pages and kind of let them float out there, or do you link to it on your sitemap or something.
Thanks in advance
Marc,
to make sure we're on the same page... different people have varying definitions of "landing pages"... are you talking about
Actually, it may not matter since the treatment is about the same.
Let's start in the middle... for long tail, you really aren't creating anything different, these are just your normal pages that are rich in optimized and visitor useful content. You would of course link into these pages from the appropriate parent pages, and quite possibly link out to related pages, etc. As for submission, all you would really want to do is include these in your xml sitemap and maybe your html sitemap if it makes sense and that pages doesn't end up with hundreds or thousands of links on it.
If you talking about PPC landing pages, it is best to exclude these through meta and/or robots.txt excludes. These pages tend to be more "ad" like interim pages to the main content and may not feature any links to them from the main site. You might set these up specifically for your PPC. SE's prefer not to index pages that cannot be reached directly from the main site. You still want these to be useful content and not spammy though.
And for the last one, don't do it. I don't think you were talking about creating spammy, keyword stuffed pages, but just wanted to make sure.
Ideally, while you may have primary category pages as well as your home page that you are targeting for high ranking in SERPs, essentially any page has the potential of being a landing page.
Sorry I meant the longtail search pages. Most of our work is finally contracts and word of mouth, but I find this pretty interesting so still play with it and try to learn when I have time.
Your responses and help are very much appreciated.
Yes, playing is good. That's always the final part of the equation with anything in SEO... there's all this stuff that then gets added to modifier that is your site. What works for one, may not work for another.
Anyway, you're welcome... just trying to give back to a community that has already given so much to me.
I'm a little bit confused about the longtail thing. Actually I get it pretty much, I think Im confused about hittail.com
Here is where I'm confused. It shows me the keyword phrases people find our site with, but wouldnt it be more beneficial to show me the possible phrases they arent finding us for?
I mean if they already found us for a given phrase were doing good. I'd be more interested in working on the phrases were NOT being found for
What am I missing?
>>It shows me the keyword phrases people find our site with
The idea is to improve on those searches. For example if you are between 5 and 10 for most long tail phrases, it will be very easy to move up a few spots on the search results with very few changes (on-page and off-page).
> I'm a little bit confused about the longtail thing. Actually I get it pretty much, I think Im confused about hittail.com
> Here is where I'm confused. It shows me the keyword phrases people find our site with, but wouldnt it be more beneficial to show me the possible phrases they arent finding us for?
> I mean if they already found us for a given phrase were doing good. I'd be more interested in working on the phrases were NOT being found for
> What am I missing?
HitTail only uses terms that are underperforming in the suggestions it issues. For example, if someone finds you several pages into the search results, and still clicks on you, it tells you 2 things:
1. You CAN be found on that term. There is actually search traffic there.
2. There are several pages of crap above you that did not satisfy the user.
Resultingly, you only probably got that particular visit by coincidence, and not as a result of targeting. Think about what happens if you deliberately target that term now. You're catapulted (very often) right to the first page of results.
Rinse and repeat.
You're not confused, you've hit on exactly the problem with keyword research in general... we all want to know what people are looking for, whether it's a million searchers or one... though we'd prefer the million ;)
And that's where I think patterns can help... we can't optimize for what we don't know, but we might be able to make some educated guesses best on what we do know... the patterns.
Using the databases helps us to find those terms and patterns that our site isn't showing up for, and looking at our own keywords on our site of course shows us what we are coming up for.
And like what I think defanjos is alluding to, because these are longer tail, less competitive searches and more importantly, people using longer tail searches are "on a mission" and tend to be more selective in what they click on, they are hunting down for exactly what they are looking for and maybe even more ready to buy... most people start searching with fewer words and add more to narrow their results. These ready to buy searchers are what we want, so if we can further optimize for these searchers and come up higher and more frequently, hurray!
Love the analysis of the structure of these long tail keywords. I'm ashamed to say I hadn't thought of doing that (and with a stats background as well)...
So you're saying it helps you improve upon what you're already doing?
Ok - cool tool and that makes sense. I was under the impression its geared towards helping you uncover those hidden gems.
Thank you for the response
Maybe the OP meant just new phrases, but I have found that improving on the longtail phrases you already rank for, is a tremendous source of traffic.
Not only that, looking at them gives you ideas for new ones you can target.
They are the hidden gems. They're the keywords that every other analytics package don't show you or burry in an undifferentiated list. Google on the concept of Keywords Forever to REALLY get the magic that's going on.
I am glad someone attempted to tackle this subject. I don't know very much about SEO, but I think it is hard to study anything like this because seems impossible to control all confounding variables.
An obvious problem is we have no way of knowing about all of the long tail searches that did NOT find the website. Maybe the study could be turned around by examining a large group of people who are all interested in the same subject or product and examining the terms that are actually entered by individual group members, and studying the various queries for similarities.
Does anyone know if keyword order makes any difference? Please let me know. Also, I am thinking that the best way to capture the long-tail is to increase the amount of relevant content. If there is an easier way, I am not above using it.
In many ways, what you are looking for isn't much different than traditional market research... identify user's needs and wants and then determine the best way to deliver on them... similar to feature-benefit statements on packaging.
The challenge here of course is scalability and how fast things move online.
And here is where I think the real distinction lies.... even more so than traditional advertising, here, users are often connecting through search by expressing their needs through their own "vocalization" whereas traditional advertising and marketing hopes to create a connection through vocalizing these needs for the consumer. Some might consider these subtle differences, but I think this is a huge distinction.
I think keyword order, like everything else, is relative. I think we are learning everyday that things vary, that search isn't one channel, but many channels dependent on the makeup of each market... competition, spam levels, networking, etc. So word order might make a difference in one area for one search, but not so much in another.
My guess would be, that more competitive markets may be more impacted by word ordering, but a huge authority site might well trump a lesser authority regardless of ordering.
But this is part of what makes this industry so freaking amazing. Once you start to figure something out and really get a grasp on it... look out 'cause it's about to change on you.
This is going to take some serious digesting!
Can you recommend any introductory articles and additional tools that help in identifying long tail key terms.
How is this data mined in the first place and more importantly how regularly is it introduced to search tools?
Really, I think it can be any tools that you use today, whether that is the keyword databases, mining the log files, running broad PPC campaigns, etc.
To some extent, the only thing different than what you are probably doing today, is in trying to look at the forest instead of the trees; instead of "this phrase" versus "that phrase" and weighing their search counts or determining which one may be less competitive to rank for, you are trying to find the patterns across the phrases, whatever those phrases may be. Just in that example there was a pattern... a simple one, but a pattern none the less... modifier + phrase.
The designers and programmers in the group probably see this much clearer... design patterns are a fundamental building block as a way for standardization and to simplify processes. Interestingly there, designers often come from a slightly different angle... often looking to utilize patterns while also maintaining originality.
The other great thing is that anyone can do this. Start looking around wherever you are right now... what patterns do you see? On your drive, walk, or ride into work... what patterns do you see? Do you make your cereal first or start the coffee maker? We aren't looking for the right or wrong, we're just looking for the 80% versus the 20%.
By tapping into patterns, hopefully you can also minimize the frequency in which this is done since those patterns are probably less likely to fluctuate over time.
Not sure about any other articles, though I'm sure there are. Likewise on the tools.
Admittedly, while the classical "digital camera" example is great for illustration here because there are some very nice patterns and so many details like model name/number, features, brands, etc., this is also a very competitive market and in reality, most of the terms listed here are extremely, extremely competitive and may be more mid-tail or even head than true long tail.
While I titled this "long tail" since that is where the most value may be found, to some extent this could be applied acrossed any search phrase.
Great article Identity! I agree wholeheartedly that "their overall value is found in total, not in individual searches." But somehow I missed the boat and never thought about extracting patterns of similar long tail searches to create new keyword lists. This is incredibly valuable. Thanks again!
Great post Identity - at the risk of sounding too newbie, I honestly didn't know anything about long tail until recently.
Thanks to the educational material in your post I decided to create my own blog post about targetting long tail and have referenced this article.
Keep up the great work - I always find your comments quite insightful!
Hey Peter, thanks for the kind words. I'm deeply honored and glad that I happened back on this post. Glad that you got something out of it.
That's one of the greatest things about this industry... there's always something new to learn! And often, re-learn, learn again, etc.
If you haven't yet, for a much more revealing discussion and education about The Long Tail, I highly encourage you to go to the source. Chris Anderson captured and put the idea down in print and you should certainly sit down and read his book when you get a chance.
Thanks - I'll be sure to check it out.