On-site search, also known as internal search, is a critical yet undervalued and underrepresented tactic in the search industry. In 2014,eConsultancy released a report that showed that 15% of companies dedicate resources to optimizing the on-site search experience, 42% fold on-site search into other online measurement responsibilities, and 42% of companies ignore on-site search. This report was released more than two years ago and after some extensive Googling, some Duck Duck Go-ing, and some slight Bing-ing, I couldn't find any reputable site releasing similar, more recent, reports.
Wait a minute... Nearly 84% of companies don't actively optimize or measure their on-site search?
Consider the following benefits of active on-site search optimization:
- On average, 30% of visitors will perform an on-site search
- When comparing revenue gained from people who performed an on-site search vs. people who did not perform an on-site search, the people who performed an on-site search generated more revenue than those who did not.
- Performing an on-site search is a strong behavioral predictor of intent to convert:
- People who perform a site-search are twice as likely to convert.
- People who perform an on-site search are more likely to return to the site with an intent to purchase.
- In the research, companies had an average overall ecommerce conversion rate of 2.77%. However, the conversion rate nearly doubled to 4.63% from people who used on-site search and found what they were looking for.
Data sources: eConsultancy, ConversionXL
It's honestly surprising that, in an industry that focuses so intently on data-driven marketing and conversion optimization, we don't put more effort into on-site search, harnessing users who are engaged and interested in our content and serves. My point is that for a tactic that can become an incredible force multiplier, there's so little information on it.
Okay, okay, you're kinda starting to rant. What are my first steps to understanding on-site search?
The first thing you need to understand before starting to optimize your on-site search is what data is being collected by analytics from your on-site search platform. For this exercise, it's irrelevant whether you're using Solr, the Google Search Appliance, or a baked-in search platform that came with your content management system.
Ultimately, the goal is to understand “Search Quality” by quantifying it:
When discussing quantifying on-site search, I tend to use the metaphor of resolution. I started playing video games as a kid and that’s continued to today. So, seeing little blocky characters evolve into something incredibly complex is a part of my experience. Clever designers were able to create experiences using only 8-bit resolution. As hardware grew more capable, 16-, 32-, and 64-bit consoles became more mainstream. Each generation of hardware supported greater resolution and the amount of visual information become more complex, more immersive, and more descriptive.
This is how I view measuring and quantifying on-site search. Start by understanding the basic metrics. Start with 8-bit resolution. This is where you’ll start seeing patterns emerge and behaviors evolve. As your experience grows, add more complexity to your metrics, look at the interplays between different types of content, look at how your on-site search results change as your content changes, look at the time it takes for people to find what they’re looking for. It won’t take too long to where you’re looking at your on-site search data at 64-bit resolution.
As your measurement resolution improves, you can understand the story deeper and with more clarity, and can use that understanding to impact greater change.
Search behavior
Search behavior is the measure of the quantitative actions a user takes on your website. This includes keywords, clicks, and calculated measurements like CTR. Measuring search behavior allows the search professional to see if any changes to the way on-site search works results in a net benefit to on-site search.
How to quanitify search behavior
The simplest metric to use to examine user behavior is CTR.
First of all, there’s an implicit assumption that clicks signal that the user has found something useful and the click is a signal of that found value. Not all clicks are indicators of search success, and there are users that click multiple links on a SERP as a way to refine their search. For now, in our 8-bit world, we need to understand that assumption, yet it’s important to measure overall CTR as a signal of search quality.
When you first start looking at SERP CTR, it’s likely yours will be low. When I first started measuring on-site search, we used the Google Search Appliance and never surpassed a 24% SERP CTR. The first question I was asked was “What’s a good on-site search CTR?” I was able to answer this by examining our different content types (blogs, videos, articles, etc). What I found through customer surveys was that a higher CTR generally correlated with a higher satisfaction with the search experience. The surprising part? Fewer people than expected had a “neutral” experience. This showed me that when it comes to on-site search, people are happy (and remember) a good search experience and people are frustrated (and remember) a poor search experience. The room for a neutral feeling has a very small window. If we had a greater than 65% CTR on our search results, we would be able to show a positive search experience and a greater chance for conversion, assuming that a good search experience meant that the user was able to find what they were looking for.
Higher-resolution metrics on CTR can include different elements of the SERP:
- Filter Section CTR
- Promoted Section CTR
- Category Result CTR
- Sponsored Result CTR
- Natural Search CTR
Understanding how all of these SERP elements influence clicks can identify opportunities for CTR optimization with either better results or a better UI. This is probably the most superficial metric to measure behavior — I haven’t even mentioned keyword refinements, revenue per search, revenue per keyword, or conversions gained through search. All are very important to measure the overall success of on-site search, but probably deserve their own article.
Search findability
Findability is the measurement of how a user finds the content they’re looking for. Findability is directly tied to the content’s rank for particular keywords. When the on-site SERP shows the results, knowing which piece of content the user clicks on, combined with the average rank of that content, can reveal if good content is being clicked, thus making it “findable.”
How to quantify findability
I take a fairly simple approach to measuring findability. I take a look at it from a content-level perspective. For each piece of content, I measure the SERP impressions, the amount of times a piece of content shows up in the top ten results. I look at the average rank, then measure the CTR of the piece of content. Using this information I can identify pieces of content that show up a lot in the SERP and have a low CTR. Looking at the average rank tells me if, on average, the content is seen. If it has an average rank of less than 10 (such as “content 2” in the table below) there’s a reason why it shows up frequently but rarely gets clicked on, despite being in a good ranking position. Conversely, when I look at “content 5,” I see that it has high impressions, low clicks, high rank, and a low CTR. I then take a look at this piece of content to see if I can make it rank a bit higher in our on-site search. If I see the CTR improve, then I’ve increased its findability as well as its value.
A higher-resolution version of findability would include the number of keywords that trigger an impression and a visibility scoring model that would describe how “visible” a piece of content is. Factoring in clicks and CTR into the visibility model would give you an overall findability score that could quickly identify valuable content that is not found and non-valuable content that is frequently found.
Result set quality
Determining result set quality is answering the question, “How good are the results for given keywords?” For example, if a common search term on your site is “shoes” and your top result is the page where the user can filter down to the right kind of shoe, and that page is in the pathway to a good conversion rate, then that’s a good result for the keyword. However, if the results for “shoes” are more general — “athletic shoes,” “red shoes,” “ballet shoes” — and the results for this keyword have a poor conversion rate, then the result set quality can be improved.
I use a fairly simple method to make sure that my results are performing according to the best interests of the company. I look at the number of times a SERP appeared and then measure the number of times that SERP resulted in a conversion or a consumption (a user reading a page that doesn’t have a call-to-action on it). Then, looking at the successful events that occur after a SERP occurs, I have a metric that I can track, trend, and show the value of on-site search.
Which metrics are core to measuring on-site search?
At the most basic level, these are the metrics we'll need to start measuring on-site search. Many of the nuances we’re measuring in SEO applies to onsite-search as well. While most of these metrics we likely already know, I want to describe them in the context of on-site search.
- Query
- A query is a package of information that is sent to the search platform. While "query" and "keyword" can generally be interchanged, keywords and keyword phrases have meaning because they're a part of language. A query is a packet of information because it is the "thing" that is sent to the search engine to be parsed, recognized, and compared against a body of documents.
- Keyword
- A keyword or keyword phrase is the representation of user intent, expressed in words. A query is an abstraction of the keyword phrase. Keywords and phrases are essentially bags of words to be compared against other, larger bags of words.
- Instance/Impression
- An instance (also impression) is the number of times or the frequency that a keyword or keyword phrase is entered into the search bar and sent to the search engine.
- Click
- In the context of on-site search, a click occurs when a piece of content returned by the search engine is clicked.
- CTR
- The click-through rate of on-site search is measured by clicks/instances. CTR is a calculated metric, but critical to understand the success of individual keywords as well as the overall health of the search engine.
- Rank
- Rank is a measurement of the position of content in the SERP. Rank can be expressed in two different ways. Rank for keyword is the number where a piece of content sits for a given keyword. Average Rank measures the average of where a particular piece of content ranks over a set of keywords.
- Examples:
- For keyword "x," document "y" is at rank 4
- For document "y," the average rank is 2.3 for keywords "a, b, c, d, & e"
- Conversion
- For content found through on-site search, a conversion is the number of times the user completed an action such as a purchase, lead gen, or consumption.
- Conversion is a quantitative metric that is binary. The user either converted or they didn't.
- Consumption
- Consumption is a qualitative measurement for content that doesn't have a clear conversion point. For sites or pages that are information-oriented (rather than conversion-oriented), it's generally described as "it is likely that the user consumed this document."
- Examples:
- A user scrolled to the bottom of the content and spent more than "x" time on page
- A user shared the document via social media
- A user commented on the document
- A user viewed this page after search and before conversion.
- Search with No Results/SNR
- A search with no results is a search that didn't match any documents, resolving in 0 results on the SERP.
- Unicorn Searches
- A keyword query that has an instance of 1 for a given time range.
That's a long list of ways to measure on-site search. How can I apply these to understanding it?
While measuring the "On-Site Search Pillars" of search behavior, search findability, & result set quality is the ultimate goal to truly understanding on-site search, the most basic pillar and arguably the most important one is search behavior. Understanding customer behavior as they interact with on-site search is likely the very first thing we'd want to start measuring to get actionable information.
Here are five ways to start understanding and optimizing your on-site search.
1: Keymatch critical keywords
Keymatching is a system that’s available in most enterprise search platforms that generally puts up to three recommended results at the very top of the SERP, right above the natural results. Keymatching is non-algorithmic and often includes a way (like Google AdWords) to match keywords to content. Usually there’s a visual element to separate it from the natural results, such as “recommended results.” Think of keymatching as Adwords for your own site. You can promote new products or increase the visibility of highly converting products. See the keymatch for “heartbleed” on Red Hat’s Customer Portal SERP. Right below the recommended results (keymatch) are the natural results:
For example, when the heartbleed vulnerability was identified, I created a keymatch for specific terms that, when searched, would trigger our keymatch system to display our Heartbleed Detector app. Based upon our search volume, we knew that people would want to check to see if they were vulnerable first, then look for a solution. We used our keymatch system to make the top of the funnel very clear.
Another strategy that’s more focused on ecommerce is to keymatch terms that match to promotions or sales you’re having. For a site that’s selling apparel where all their socks are on sale, keymatching will give the user the information that all socks are discounted.
2: Get a good overview of the health of your on-site search.
To do this, look at the overall on-site search CTR. Don't worry about what it is at the moment; this is about understanding how well things are performing in the real world. Use this metric to get a strategic view of how search is performing. If you see that your on-site search CTR is lower than you want it to be, look at the top keywords searched, then repeat the searches on your on-site search.
It's very likely that your top searches are "high volume but low intent" keywords. You might be looking at top-of-funnel keywords like "shoes," "insurance plans," or "sci-fi movies." Is there a good landing page on your site so that the user can funnel down to what they're looking for? Generally, these low-intent keywords are from people who may not know what they're looking for, but will know it when they see it.
Use either a keymatch system or a boosting system to push top-of-funnel pages to the top of the search results. If you don't have these types of pages, you'll know that you're going to need them. What’s a boosting system, you ask? Boosting is a term that adds relevancy to a piece of content or even content types to rank higher in the natural SERP.
For example, imagine that the term “red shoes” resulted in on-site natural results for different types of red shoes. However, data shows that people looking for “red shoes” convert at a much higher rate for the imaginary “Red” brand that has a shoe line. One way we can improve the natural search results for this scenario is to boost the brand “Red” for queries including “red shoes.” While you can force content to the very top, keep in mind that boosting is a broadsword solution that may cause collateral search damage and unintended consequences to your search results. However, if the data supports such a broad change, using a boost in your on-site search platform is a good tool to use smartly.
3: Look at individual pieces of content in the context of on-site search performance.
From the perspective of an individual piece of content, it will likely show up for a variety of keywords. Looking at the data from on-site search at the content level will reveal what keywords trigger that piece of content to show up in the top 10 results.
The first step is to look at how much traffic that piece of content gets from on-site search. Secondly, look at the clicks and CTR. Notice that in the following example, most keywords deliver a decent CTR except one. This piece of content delivers no clicks, no revenue, and ultimately no value to the user. This is a powerful signal to the business to look at why this piece of content ranks for that keyword. Sometimes, on-site search can deliver poor or irrelevant results. The solution here is to either fix the content so it delivers value or use your ability to de-boost that piece of content for that keyword.
Clearly, this content works with other keywords, so the problem may not be the content, but the fact that it ranks for a keyword that has no relevance to the content. To understand on-site search’s weight in generating revenue, compare the revenue and conversion rates with other marketing channels such as SEO, email, direct, or social.
4: Use searches with no results to enhance content
If you have people searching for things that deliver no results and the frequency is significant, this is a great way to give your content team information on what to write about next. For example, the site ThinkGeek had an April Fool's joke about a Tauntaun sleeping bag. I talked to someone at ThinkGeek and they mentioned that even though the product wasn't real, they started seeing several hundreds of searches for that product on their site. What was surprising to them was that the searches not only increased, but continued well after the April Fool’s joke. As a result, they actually made that sleeping bag and it became a customer hit. While it wasn't only found via on-site search, the consistency of mentions in social media, on-site search, and customer service convinced them to actually make it.
5: When it comes to UI/UX, test snippet UI, test SERP UI, test everything
Don't accept the out-of-the-box search UI. Test putting images or embedded videos into the snippet. Test putting linkable "top level" breadcrumb navigation into the snippet. Create good A/B tests to see what will increase clicks and conversions. Talk to internal stakeholders, talk to your users, and find out how they use search. Once you know what your critical business goals are around on-site search, testing the UI can significantly increase CTR and conversions.
Great, thanks… Now I want to do all the things. Where do I start?
I know that I’ve covered a lot of topics that really require a lot of nuance, a lot of industry-specific idiosyncrasies, and other considerations to on-site search. Because there’s not a lot of current information or even communities that surround on the non-development side of on-site search, I can understand that there might be questions about how do I start using this information.
My advice: start by understanding what your search platform is. Do you use IBM Websphere for your ecommerce platform? Do you use Apache Solr? Google Search Appliance? Elasticsearch? Look deeply to what the features of your ecommerce or search platform has. If you haven’t already, hook up your analytics platform to your on-site search and start collecting data. I’ve found that many times, when companies have set up their analytics, they don’t apply basic metrics to their on-site search. Look at the data you have and start setting up experiments. Try to make a piece of content rank higher in your on-site search. Experiment with meta-keyword tags (yes… Google and other search engines ignore it, but generally, on-site search engines still love meta keywords). Start small experiments to prove that you can increase revenue by making small changes. Once you understand your search platform, once you’re seeing small changes you’ve made that have positive results, dig deeper. Share your experiences (I’d love to hear them) and find people to collaborate with. I cannot claim that I know everything, but what I hope to have done is created a small group of enthusiastic people who can work together to create even more information on this oft-ignored, but oh-so-useful subject.
Some acknowledgements:
Red Hat isn’t just about open source technology, it’s about being an open organization. With that open and collaborative mindset, I want to share my thanks and deep appreciation to some people who helped me with this article. Linda Caplinger, Jairus Mitchell, & Thomas Stokes are brilliant people who provided feedback, suggestions, and welcome critiques.
If the site uses Google Analytics, it's just a couple of clicks to start collecting the essential data needed to start assessing these issues/opportunities.
In the Dashboard Admin, navigate to the bottom of your Analytics View's View Settings and set the Site Search Tracking button to On.
Then add the query parameter that identifies the search string on your site. For standard WordPress for example, it's just the letter "s" (no quotation marks).
I'm always amazed that most SEOs ignore their own mini-search-engine and its SERPs! You've provided some terrific additional food for thought here too - thanks, JP!
I always find it interesting on eCommerce sites to simply compare the value of visitors who use internal search vs those who don't. The usually significant difference is often all it takes to convince clients to put some effort into improving the quality of their on-site search results. (Hint - the default internal search tool in WordPress sucks rocks.)
Paul
Thanks! You're absolutely right. In GA, it's really easy to set up onsite search tracking. I'm using Adobe Analytics, so it takes a bit more effort to get more of the nuanced data like refinements and time on SERP, but it's totally doable and once done, very enlightening.
Cheers!
We intentionally don't use GA for some of our sites, JP, relying instead on Piwik and other solutions, so I found the article really useful. Many thanks!
(We've used Freefind for many years for in-site search on non-WP sites)
ooooh, I don't know Piwik, looks like i need to read up on that. thanks for that new bit of information.
cheers!
Very interesting point. I didn't know it at all :)
Thanks a lot!
I'd love to get a head-count of people who regularly, rarely or never use onsite search data.
- If you do use onsite search info, what are the top things you use it for?
- Do you have dedicated resources or is it a responsibility tacked onto the search professional?
- Do you use "out of the box" KPIs or metrics or have you customized search data to your unique needs?
- Do you report onsite search information regularly and what do you report on?
- How is onsite search's value perceived internally?
Hi JP!
First of all, really nice post, I enjoyed with it a lot ;)
For me, I know that on-site search it's really important, on my projetcts I try to focus on it specially on eCommerce, but, being honest, I don't dedicate to much time on blog sites...
Thanks a lot ;)
I used to be an in-house for an international bike retailer, this is where I really started getting into on-site search, mainly because it helped with determining promotions.... but then I kind of went down a rabbit hole, applying my SEO skills to it and found that it really helped. Then I realized that there wasn't a lot of content about it, so I learned more and figured that I should probably share :)
thanks for the response! cheers!
I am sad to admit that we do not have site search in the traditional form. Instead we have a virtual agent which most people think is live chat and therefore not very satisfied with their experience.
Those that do visit our site frequently and understand that it isn't a real person are able to get what the need from the product, but also know it is very limited. We do not really monitor the data since we have let go the analyst that had this job. It is also part of another teams responsibilities that do not have the time to maintain or improve the product.
With that said this year we are moving the responsibilities over to my team and we will be working with the vendor to improve the experience and we are going to be adding site search.
Cheers JP, likewise to many of the other commentors, I also havent' touched onsite search.
I access it when I can, I run PPC here. Not every client or brand has collected this in the past. Use it to help figure out what the customers want or don't want from the site. Great customer intent data.
Great post, JP. Site search is so key in an overall site and marketing strategy, I'm surprised it doesn't get as much attention as it does!
It's also a good idea to filter out those within the company who are performing searches so that it's excluded from the search data whether through IP filtering or if using Wordpress, filtering logged in users.
Great point. At Red Hat, we've got several thousand employees across the world, so part of my job is to look at the search differences between internal associates and customers - we've found that when it comes to associate vs. customer search the behaviors are different. It's a fun challenge to test changes to SERP and results to see which group benefits from those changes. From a strategic perspective, I try to focus on the customer experience improving, but keeping both groups in mind is absolutely critical.
Even as a Paid Media person I've found on-site search very helpful. It's a great way to look at and understand what keyword you want to bid on or how you want to write your ads. At least for clients that have on-site search; I always ask for this data if it's not something I can easily get in their GA accounts.
Very informative blog, JP! To be honest, I never really paid a lot of attention to onsite optimization as my focus has been on SEO. Will definitely do some research and testing on onsite search and get back to you!
I'm generally on twitter (@jpsherman) where you can feel free to bug me w/ questions - I'm always looking to find more people who are interested in this so information can be grown and shared.
Hi JP,
You have taken a very interesting topic for discussion.Actually, an on-site search is a pivotal part of a site but it has given less priority by website owner where it should be given full importance. It should be highly optimized so that it could bring easiness in searching something in your site by users and also it increases user experience as well. The one who is not so serious about on-site search and if that person is reading this post, I bet that person is going to fully utilize his/her time on on-site search and make their site more interesting and descriptive.
Good post, JP. I had never thought of the importance of onsite seach. But now I have many ideas and tools to improve the onsite search of my websites and the sites of my clients, If I have doubts in the process, now I know who to ask, haha. Thank you very much! :)
thanks!! hit me up on twitter (@jpsherman) i'm always happy to chat about this, thanks for the response.
The more we facilitate things to our visits to get the action we want to get more will grow our conversion rate. And if our website is an electronic commerce it is imperative to have well worked "quick search". We all like to find what we want at first and not having to walk around a website. We must always think about our clients, what are you looking for? what do you need? How do you like to get it? What language do you use? ... and not just Google, this will also improve our conversion rate.
Is it enough with the search bar in wordpress or would you take more actions to improve on site search?
Hello JP
This was an informative and useful article. Some of the genuine and unbiased advice easily explained. I have been working on SEO with many businesses, small and large, for many years to help them grow their business in an honest and powerful way. Still, I have learned a lot new from your post. I look forward to more comprehensive and insightful posts like this in the future.
Cheers,
Cody Oelker
Thanks Cody!
I'm already working on my next one :)
Cheers!
I don't have enough content for onsite search to be useful for visitors at the moment :)
however, what I'd do is look at what your users are searching for and create content based upon their queries - you get relevant content, a better seo presence and happier future customers. :)
Cheers!
Not sure if you remember, but on-site search was the foundation of the Triangle's ascent into the tech community. Jud Bowman's first company, Pinpoint (which became Motricity) was initially focused on this area back in 1999.
wow. I'd heard about that when I worked at a large seo agency in morrisville - but never made that connection when i was just a search newbie :)
Great article. No doubt that Site Search presents a huge opportunity for customer experience differentiation today. However, this is hard. It requires deep technical (search) expertise such as relevancy modelling, coupled with not only a great understanding of analytics as presented here but also search user journeys and intent. It is therefore both an IT problem and broader UX problem, plus an ongoing search management problem. This means constant optimisation to always be on top of user expectations, just like Google (though this is invisible to us all). The search technology is largely irrelevant - commodity in fact. What we are also now doing at Realise is building on search further using cognitive computing to automate other aspects of customer service - starting to provide answers and results. This can also then be presented on the roadmap using Conversational UX (Chatbots, Alexa).
You are so right. It does take a pretty strong and diversely talented team. You need dev & UI/UX support to make changes (we use Solr) we have a search/ behavioral analyst and we had a statistician donate some of his time to run A/B & multivariate tests. This goes back to my point about scaling resolution. It's possible to make good changes at the 8-bit level, and when you start getting statistician support, you're hitting 32-bit resolution :) and yet, there's so much more that can be done. There's a field in cognitive psychology that involves studying the interaction between technology and humans - but, i like the fact that a company can make good changes over time with a small team. Thank you so much for your perspective. Cheers!
Nice article on a topic dear to my heart. Another tactic we find useful is to look at not only the searches with no results, but also the searches with poor results, ie popular, but with a low CTR.
Also the other obvious use for this data is for keyword discovery for SEO . If people are using a term on your site then they're probably using it on Google as well.
absolutely! that's a really important point. thank you!
Hi Jp, Great articles, I am new in seo but trying to rank my site. Similarly, i am reading moz regularly and getting a lot help from this site. My site is about celebrity, how to optimize it for on site search?