The past few years have seen an explosion of usability and Conversion Rate Optimization (CRO) tools hit the market. There have been many good roundup posts about these tools, but I want to focus today on a more in-depth approach to putting just 3 of these tools to work: (1) Five Second Test, (2) Crazy Egg, and (3) UserTesting.com. Total cost to do one round of testing: $224.
(1) Five Second Test ($20)
The premise behind Five Second Test is incredibly simple – show a visitor your site for 5 seconds and see what they remember (or, alternatively, where they click). This is a great starting point for getting some starter observations about your visitors.
How It Works
Setup is easy – just submit a screenshot of your web page or prototype (great for design comparisons) and the replies start coming in. You can view them individually or grouped by concepts. Five Second Test is actually free, but the $20/month package means you'll get a larger response rate. It's worth the extra cash, IMO. You can also earn credits ("karma") by taking other people's tests – it's kind of fun and can be informative.
What to Test
Think about the kind of things you want your visitors to know about in 5 seconds: The big questions: Who, What, Why. Here are a few uses I recommend:
- Do visitors recognize your brand?
- Do people get what you do?
- Is your tagline descriptive and effective?
- Is your page too visually noisy?
- Is Concept B better than Concept A?
- Can people find your call to action?
If people are remembering things like "blue", "blonde girl", and "ugly site", you know you've got some work to do (those aren't far from real examples of what I've seen).
(2) Crazy Egg ($9)
Heat-mapping tools like Crazy Egg take user activity and translate it into visual maps, helping you to easily visualize how people interact with your site. Crazy Egg was founded by SEO wonder kid Neil Patel, and is an amazing bargain at $9/month. If you can't bother to spend $9 on improving your website, feel free to stop reading this post. I'm serious – go buy a Venti Iced Mocha and a cookie instead of spending money on your business.
How It Works
This one's a little bit trickier – you'll have to install a JavaScript snippet similar to Google Analytics and other tools. Then, Crazy Egg starts tracking clicks on your specified page (try to stick to one page, as jumping pages can produce odd results).
What to Test
Crazy Egg not only allows you create to visual heat maps, but also has a "confetti" mode that lets you visualize clicks by segments, such as referring sources and new vs. returning visitors. Here are a few questions a heat-mapping tool can help you answer:
- Are people clicking where you want them to click?
- Is your navigation effective?
- Do you have too many choices?
- Do search visitors behave differently?
- Is your call to action getting clicks?
Although some heat-mapping tools can get bogged down in the visuals, I think that Crazy Egg has a very simple, elegant reporting approach that can give you solid insights quickly. Once you've gathered some initial impressions from Five Second Test and Crazy Egg, it's time to do some real user testing...
(3) UserTesting.com ($195)
It used to be that user testing required a lab, expensive equipment, and a difficult recruiting process. Now, you can use remote testing services like UserTesting.com to get quick, inexpensive user feedback. While I won't say it compares apples-to-apples to laboratory testing, I often find that the insights from even a handful of remote testing subjects can be incredibly useful.
How It Works
Setup is pretty straightforward, but doing it right can take a little bit of time. Technically, you just need to submit your URL and a few instructions to visitors. You pay $39 per visitor and receive both written feedback and an online video of the user walking through your site (with voice-over). Although this is a topic of some debate in the usability community, 5 users is a good number for uncovering core insights and getting solid bang for your buck.
What to Test
Take some time setting up your questions. Traditional usability tests are task-oriented – you tell someone to try to complete a task in a fairly open-ended fashion and watch them go to work. Be specific about the task and ask follow-up questions, like "Would you trust this site enough to make a purchase?" (I generally ask 3-4 follow-ups). A few questions this kind of qualitative testing can help you answer:
- Can people complete the task?
- How long does task completion take?
- Do users experience common stumbling blocks?
- What are visitors thinking out loud about?
- Does your search/navigation work as expected?
- Are you missing features people might be looking for?
- Do visitors get frustrated using your site?
Qualitative testing can be a great precursor to quantitative (A/B and multivariate) testing. Don't throw design changes at the wall and see what sticks – put user testing to work to uncover hidden issues on your site. We all need a fresh pair (or 5 pairs) of eyes from time to time.
Here's to $224 Well Spent
I'm an entrepreneur and a Bohemian – I understand that parting with money isn't easy. The insights you'll gain from just over $200, though, will, in my experience, easily yield 10X or even 100X back in online sales improvement. Solid qualitative data collection will also prevent you from making costly mistakes and will better inform how you look at your analytics and quantitative testing. There are plenty of good tools out there – choose a couple of them, and really put the effort into understanding how they work. You'll be well rewarded.
Update: We just published a YOUmoz post about Crazy Egg that should be an interesting read for anyone who enjoyed this article. David gives some nice examples and a case study of how heat-mapping got one of his clients an 87% conversion boost.
Another addition for what to test with UserTesting - your competitors. If you can gauge what people like and dislike about their sites, it can have more of an impact on your site than the testing can (speaking from experience!).
Sounds like a great YOUmoz post Greg. I'd be interested in reading about it.
Great point - actually many of the tools out there now (maybe 2/3, if I had to guess from memory), allow you to use either just the URL or a screenshot, without installing any code on the site. This makes them great for competitive analysis.
Interesting idea - I think our users would be impressed with your website, but I'd be interested to know what your testers thought of ours... ;-)
A really interesting post. Insightful into what tools are out there that you can use to actually get better feedback on your site. I think the user feedback is on the pricey side considering it`s per user and although you are getting dedicated test users using your site, could the same not be achieved through colleagues or friends? I understand the feedback would be more basic probably but why over-complicate things?
CharlieB29, I don't think Dr. Pete has any friends...
Ouch. I mean, sure, it's true, but it still hurts, man. My cat is a lousy test subject - she can't right-click to save her life. Bad kitty.
Some of us are just anti-social :) Honestly, I sometimes find using a 3rd-party service a bit easier than corralling friends and family, but you can definitely test with people you know and get useful insights. The only trade-off is that there are different biases to the two approaches. Ideally, I'd have someone do a little of both.
Dr. Pete.
In regard to UserTesting.com and five users providing valid results, as you indicated this is indeed controversial in the usability community. My opinion is that five users provides a good number if you don't get any outliers. But given that is it challenging to know if you have an outlier with just five subjects, six to eight would be far better.
However, thanks for providing a good review of three useful and valued priced tools/
I have to admit that I'm not a good judge of the general public's ability to interpret a test result, but I think it really boils down to process and patterns. If you see someone engage in a process and you go "Wow - I never thought of the site/navigation/etc. that way!" - you'll almost always learn something from it (i.e. it almost can't be an outlier, within reason). Likewise, if 2-3 people run into the same issue (out of 5, especially), you've got something worth fixing.
The thing I try never to get bent out of shape about with remote user testing is opinions. If someone says "I hate this" or "This color is ugly", oh well - that's their opinion. If someone can't figure out what to click to add something to their shopping cart, that almost always matters.
I should also note that I see this as a qualitative/discovery process, not a quantitative one. You don't take one user and make a major, costly change to your site. You can take one user to reveal a potential issue or need and then test that with quantitative tools.
I like your observation about colors and opinions versus usability. I don't focus a lot on CRO (in part because I find the subject overwhelming, in part because there's only so many hours in a day). But you are right - not liking the color green doesn't make it hard to use the site. Not being able to find the add to cart button makes it hard to use the site.
Thank you for this moment of clarity at 5pm on a Thursday afternoon...:) I think I can apply myself a little better now to all the reading I have to do on the subject.
Nice post!
You have now created an online, easy to use, quasi-focus group for less than the room rental cost at my local hotel. All without the giant piece of one way glass and the guy with the clipboard. Not to mention Crazy Egg can be used without the visitor knowing they are part of an experiment, which in my mind increases the relevancy.
Good job, that's why you are the doctor and I'm not :-p
I have tried heat mapping and here are some results.
1, People don't really care much about content below fold
2, People are drawn and clicking to visually different elements of websites (action buttons)
3, People click on footer links (not many but some still do)
4, If you show them only one action button of course they will click on it (even if the visual is shit)
I was able to fix few issues on a website with data gathered from heat maps.
I realized that sometimes I do things poorly and it is my mistake not a lack of users intelligence.
I believe that CRO is very important. Every SEO should give a thought about it. There is no point getting traffic to a website and than waste it by having badly created website.
"People don't really care much about content below the fold"
Well, maybe. CRO and website pages are really married to the content. If the content is compelling, then you get results like this...
https://www.conversion-rate-experts.com/seomoz-case-study/
(Note the length of the page compared to the original)
There are a lot of correlations between the content listed first and the "heatmaps" generated because the most compelling content and actionable buttons tend to be higher on the page, but that doesn't mean that people don't care about the stuff below the fold. I know you point that out as a generalization, but it can be a little dangerous to make such a strong assumption, so I thought I would chime in.
Also, the one action button comment I tend to disagree with. Every page has at least two inherited action buttons - the "back" button and the "close browser" button. If your calls to action or content is weak or not compelling, they will use the other action buttons and "bounce".
A/B testing, web analytics and conversion optimization will have a great impact on almost 99% of all online businesses - but I get surprised EVERY time I see A/B testing, content strategies, data driven decision making, search strategies, social media strategies, website architecture and usability strategies, keyword strategies, public relations, print advertising, and so on in COMBINATION.
Another challenge is that (most) designers don't understand web development and web developers don't understand design, usability and accessibility. The content writers don't know anything about SEO and the CEO only seems to be interested in hit-counters, visits and page views.
Welcome to the Online Business Prevention Unit! :-D
One thing I find about qualitative testing is that it can be a great door-opener for A/B testing and analytics. Show someone a problem with their site visually or show them a real user stumbling, and it can really open their eyes. We can preach at them all day long, but they'll always have a defense against the experts. One visitor who can't buy something speaks volumes.
I knew I saw these tools listed somewhere. Here is a similar resource written back in January by Avinash
https://www.kaushik.net/avinash/2010/01/sweet-web-analytics-resolutions-kick-notch.html
Dr Pete also did a pretty good rundown a while ago: https://www.usereffect.com/topic/guide-to-low-cost-usability-tools
Full Disclaimer: Dr Pete in no way paid, coerced or threatened me into this comment, I'm just a big fan :)
Oh, absolutely - I linked to 3 lists (admittedly, my own included), but there are plenty of good ones out there. There are actually so many tools and lists of Top 10-20 tools that I think people often don't know where to begin.
Avinash is actually involved with another great tool - 4Q Survey - that's well worth checking out (and is free).
Thank for the 4Q-Survey suggestion.
The integration it has with Google Analytics is a big thumb up... as to know that Avinash is involved in this project.
Great roundup of tools that certainly are much cheaper than having an in house focus group session - I am sure you will get infinately better data with these tools than the development department doing their own user testing :)
One question - with the last option, are the users that are on there more likely to know what they are looking for on your site if they are doing this type of thing more regularly than 'average users'? Thus skewing results?
You have some limited control over who you get, including how savvy there are, but yes, there's always an element of self-selection. If you're well-trained enough to record your own browsing session, you're not a complete internet noob. The other low-budget testing element is the DIY approach - get a couple of friends or family members that are not internet savvy (harder to find these days), and sit them down in front of your site. Before you do, buy a copy of Steve Krug's "Don't Make Me Think" for $20. Another excellent investment.
Dear Pete,
it seems that your nemesis - being published always after or before a Rand post - is still there :)
Thanks for this actionable post, that combines good tools/services with a budget small businesses can afford (or that can not scare small business clients).
I personally use also ClickTale. I know, the price for small sites is 10x the Crazyeggs one, but the insights it gives are very very useful in order to test the usability of your site.
Finally, I think that this post [Note: it's not mine] about this same topic (usability tests that does not mean selling your son for paying them) can be inspiring. For my italians fellows here, this is the italian version of same post.
That was a Great List Apart article you linked to G. Somehow I missed it. The sub-titles were hilarious:
Method #1: if you like people Method #2: if you like machines better than people Method #3: if you like people, just not talking with them
Humm. Here we are talking about $224*12= $2688 of added expenses/annum/website for a possible yield of 10X or 100X in online sales improvement. While there is no gaurantee that using these tools will improve your sales to that extent, there is 100% gaurantee that you will be shelling out $224/month/domain. This amount is much more than the average salary of an in-house seo/webmaster in most developing countries. So this is certainly not a small amount and not financially viable for small websites/businesses. I wouldn't have raised this issue had this blog catered to only US and UK audience. If you can increase your sales by even £200/month from your website just by spending £145/month on site usability, then it is worth spending. But if you ask someone from Asia to spend this much amount on site usability, he would laugh and would prefer to hire half dozen guys instead. This issue is also important because asia and africa are emerging markets and provide tremendous opportunity of growth esp. countries like India and China which may one day have more internet users than the rest of the world. So it is a request to seomoz to think about the people in other parts of the world tool. You guys easily ask to spend couple of hundred dollars here and there like it is nothing.
Here is my CRO advice for almost $0
Alternative to five second test
Ask your friends/acquantances who have never seen your website to tell what they make out from your website. Test what Dr. pete has specified in the post above. Give them 5 seconds for this. Note down their responses. Even if you don't have any friend, you will always find someone near your office/company who could spare 5 seconds for you. If you can't do even that, then buy the five second test tool.
Alternative to Crazy egg
Our good old Google Analytics is the alternative. Google analytics event tracking feature can track:
Are people clicking where you want them to click?Is your call to action getting clicks?.
Google Analytics internal site search feature can track:
Is your navigation effective?Do search visitors behave differently?
And then we have the siteoverlay feature of Google Analytics. Not as good as crazy egg but then something is better than nothing.
Alternative to usertesting.com
Do it yourself usability testing without hiring test participants. Take feedback from your clients/visitors and record their objections: regarding navigation, web design, usablity, checkout process etc. In this way you know exactly what to test and what should be test first. This eliminate the need of getting test participants in the first place. Every website has tons of usability issues and you can't fix them all at once. Therefore it is very important that you test those issues first which have significant impact on conversions and user engagement and who can tell you better about these issues than your client/visitor. The only problem you will face here is getting feedback which is not so easy. You need to find innovative ways to get feedback from your visitors/clients. One good way is 'thank you' call. Call/email your client when he makes a purchase and take his feedback about the shopping experience, site navigation etc.
Your point about doing user testing on the cheap is well taken Himanshu. Certainly, if you don't have the budget then $224 is too much. But assuming this is customer work you are doing, you would just build it into the fee you charge them. And if you were restricted still, then you would have to make do with less than Pete recommended.
Personally, if I only had a few dollars, I would start with Crazy Egg. There is nothing like seeing a colored heat map for instant recognition of how users are using your site.
Spending someone else money is never a problem. Problem is justifying the spend. There are countless tools on the net for analyzing this and that. Majority of them provide stand alone functionality/implication of Google Analytics or simply your common sense. I am not against these tools. If you have the budget, then go ahead and buy them.
Just an alternative angle on this, as a contractor (not an in-house) - I've actually used Five Second Test before a contract was signed and sealed to help make my point to clients. That small spend in one case results in a decent-sized contract. Also potentially doable with any tool where you don't need direct site access (Crazy Egg isn't viable for that use).
Although two of these tools are per-month, I'm actually only suggesting a one-time, one-month spend. That's not really what the per-month tool providers have in mind, and I personally have an ongoing subscription to Crazy Egg because I like it so much, but you don't need to do this for a year to see results. Often, those initial insights can have a huge impact and prevent many costly mistakes. I've seen huge ROI on this approach.
Self-testing is certainly an option, although more time intensive. I like UserTesting.com for clients, because I can white-label the resulting videos - much harder to DIY as a consultant, but more viable for in-house. In that case, I highly recommend buying or borrowing a copy of Steve Krug's book "Don't Make Me Think", which gives great, practical advice for doing your own user testing. It's hard to mimic the Five Second Test functionality properly, but as I mentioned, there's also a free version, so just try that first. Then spend $9 for one month of Crazy Egg and use it on your home-page.
Obviously, though, the $9 Mocha + cookie reference was aimed at the US audience. We tend to complain about not having money to spend on our businesses (ESPECIALLY on the web) and then go pay $30 for 2 movie tickets, popcorn and Coke. That attitude frustrates me a lot.
Thanks for clarifying peter.However usability testing is not one time activity, so recurring cost is inevitable. I would also recommend 'rocket surgery made easy by steve krug' for usability testing and 'landing page optimization' by Tim Ash. These books are worth every penny and best investment for any seo.
I defintiely see your point, but to someone with the available funds these are very helpful tools and well worth the costs IMO. I definitely look forward to using crazyegg as I hadn't known about that before this post and the earlier youmoz post by Dave. Another potential heatmapping tool to consider is clickdensity.co.uk. It does have a cost associated with it, but it has a free trial for those interested in trying it out. I am not affiliated with them in any way shape or form just wanted to give another free tool (at least temporarily) that could accomplish a simliar task.
Great summary of these products. How are customers move around, react, buy or leave is so important.
Great post! We are currently using all those tools and finding them very effective and affordable in our CRO.
Just ran a 5-person test on UserTesting.com and very pleased with the results.
Tim Ash's Attention Wizard is another tool you can consider for CRO.
Absolutely - I had a great chat with Tim at the SEOmoz PRO seminar last week, and his team is up to some cool stuff with Attention Wizard.
I should make it clear that my intent was just to focus on a process and use these 3 tools to illustrate it. There are plenty of other good tools out there, some of which I use regularly.
The problem with the Five Second Test is that it's all usability experts who will likely be using it. If you want the average web visitor, you'll need to go elsewhere... unless there's something I'm missing?
It's probably true that most of the Five Second Test test subjects are web savvy (designers, developers, etc.). In this particular case, though, I don' think it make a huge difference. You're not really looking for their thought process or ability to navigate the web so much as first impressions. Even the savvy among us can only process so much in five seconds.
Great Post, I used CrazyEgg before its realy cool tool, never used Five Second Test, but it looks great and usefull most visitors will spend 2-5 seconds if you can't get their attencion then they are gone,
Marios
Great tools but it's little bit cheaper for in house use
CRO always seems to be the best kept secret of successful campaigns.
Great Article Dr. Pete! I've used one other platform recently called Feedback Army. Affordable service and good feedback. The low quality feedback were less than 5%. Their guys seem to be all over the world - Eastern Europe, India, China, etc.
Guys, you might consider that as an option. Thanks once again for the great article!
Dr. Pete,
I've always enjoyed your posts but I gotta tell ya...this is one of my recent favorites. I know in my experience, as I've started to focus on CRO a bit more, clients are in love with this idea.
I mostly use google website optimizer for A/B testing but that FiveSecondTest looks great! I had never come across that. Even participating in other ppl's tests has been really valueable.
Thanks for the great post and I'd love to continue to see more on CRO...
Great tool and great tips - very helpful stuff.
I knew I saw these tools listed somewhere.
I really like the idea of all three tools and the prices are great too.
The 5 second test I think is very important, great post cheers
Hmmm... and you said priceless, yet it has a price apparently :)
I was slightly afraid people would look at the title and think I was trying to sell them something:
"Dr. Pete will paint any car for JUST $99.95!"
You have soooo just acquired the nickname Earl. Do they have Mr. Scheib where you hail from?
Really nice post Dr. Pete. Simple and actionable. User Testing is a great concept. They used to be even cheaper which was nice. But still, compared to a formal usability experiment, they are a fantastic value. The one concern I have with them is the same as Dan Taylors above. I've always wondered if being a "professional" usability reviewer affects their judgments.
Thanks for sharing those tools. If you want the feedback, is not it easier to put a comment box on the website ?
Great post. Signed up for both Crazy Egg and 5 Second Test. Can't beat the price either.
A very timely post, Pete. Cheers. I hadn't heard of Five Second Test, so thanks again.
Liam
Thanks for the post,
it couldn't come at a better time, we are just in the middle of redesigning our website and as a conversion freak, I put our mock up to the test right away.
Am I allow to put my tests here to get feedbacks?
Break it down! It doesn't get much easier. Thanks for spelling it out for us.
I know some people have voiced concerns over the cost of usertesting.com, but from personal experience, at least for my company, it's well worth the price.
On the first round of testing we did for our website we found HUGE problems that we not only didn't know about, but the things causing the problems we thought were helping customers!
What was even more impressive was how much insight we gained on the second round of testing, after fixing all the issues from the first round. I was certain that we got rid of all the problems after that first test and was ready to declare Mission Accomplished.
We're now on our 5th round of testing and still finding lots to improve on our site!
If you're worried about initial costs when you're just dipping your toe in the water, or if funds are limited, test with just 3 users per round. That's been our approach, and it's worked great. If it doesn't work for you, just add 2 more users to the same test.
Thanks again Dr Pete, this is a great list of resources to try!
Don't Forget Litetest also another customer feedback source for your site and free :)