Last week, the New York Attorney General's "Operation Clean Turf" fined 19 companies a total of $350,000 for writing fake reviews on behalf of their clients. The case sets a laudable precedent not only for the future of local search, but for digital marketing more broadly.
While the amount of the fines is hardly Earth-shattering, the outcome of this operation should give pause to any SEO or reputation-management company considering quick-and-dirty, underhanded tactics to boost their clients' rankings, "improve" their clients' reputations, or launch negative attacks on competitors.
In the wake of this settlement, however, a wave of media coverage and a study by researchers at the Harvard Business School have clouded the reality of Yelp's review filter—already poorly understood by typical business owners—even further. In this piece I hope to dispel four misconceptions that it would be easy to conclude from these recent publications.
Likely elements of review filters
Review characteristics
- Use of extreme adjectives or profanity in the review
- Overuse of keywords in the review
- Inclusion of links in the review
- 1-star or 5-star rating (see discussion of HBS study below)
User characteristics
- Total number of reviews a user has left on the site
- Distribution of ratings across all of a user's reviews
- Distribution of business types among all of a user's reviews
- Frequency of reviews that a user has left on the site
- IP address(es) of the user when leaving reviews
Business characteristics
- A sudden burst of reviews preceded by or followed by a long lull between them.
- Referring URL string to business page (or lack thereof)
1. "Most aggressive" review filter ≠ "most successful" review filter
Yelp representatives made little effort to contain their glee at being cited by the NYAG as having the "most aggressive filter" of well-known local review sites. In an interview with Fortune, Yelp's corporate communications VP spun this statement by the NYAG as validation that his company's filter was "presumably the most progressive and successful."
As I stated in the same Fortune story, I agree 100% with the NYAG that Yelp's filter is indeed the most aggressive. Unfortunately, this aggressiveness leads, in my experience, to a far higher percentage of false positives—i.e. legitimate reviews that end up being filtered—than the review filters on other sites.
Google, for example, has struggled for almost as long as Yelp to find the perfect balance between algorithmic aggression and giving users (and indirectly, business owners) the benefit of the doubt on "suspicious" reviews. Now that a Google+ account is required to leave a review of a business, I suspect that the corresponding search history and social data of these accounts give Google a huge leg up on Yelp in identifying truly fraudulent reviews.
I'm not necessarily saying that Google, TripAdvisor, Yahoo, or any other search engine presents the most representative review corpus, but it's a pretty big stretch for Yelp to equate aggression with success.
2. "Filtered reviews" ≠ "fraudulent reviews"
To Yelp's credit, even they admit that legitimate reviews are sometimes filtered out by their algorithm. But you sure wouldn't know it by reading a recently published study by the Harvard Business School.
In a throwaway line that would be easy to miss, the authors state that they "focus on reviews that Yelp's algorithmic indicator has identified as fraudulent. Using this proxy…" they go on to draw four—actually five—conclusions about "fraudulent" reviews:
- Their star ratings tend to be more extreme than other reviews.
- They tend to appear more often at restaurants with few reviews or negative reviews.
- They tend to appear more often on independent restaurants rather than chains.
- They tend to appear more in competitive markets.
-
"Fraudulent" 5-star reviews tend to appear more on claimed Yelp pages than unclaimed ones.
I would only highlight my friend Joanne Rollins' Yelp page, and thousands of other business owners' pages just like her, as qualitative evidence to rebut their logic. I don't dispute that Yelp's review filter is directionally accurate, but it's crazy to assume it's anywhere near foolproof to use it as a foundation for a study like this. It leads to self-fulfilling prophecy.
In fact, there are five very easy explanations of their conclusions that in no way lead you to believe that the overlap between filtered reviews and fraudulent reviews is even close.
- Yelp uses star rating as part of its filtering algorithm. This is an interesting finding, but not applicable to "fraudulent" reviews.
- Restaurants with few reviews or negative reviews are engaging in proactive reputation management by asking customers with positive experiences to review them. This is simply a best practice of online marketing. While it violates Yelp's guidelines, by no means does it indicate that the reviews generated by these campaigns are fraudulent.
- Independent restaurants tend to be much more engaged in online marketing than chains. Speaking from years of personal experience, chains have by-and-large been very slow to adopt local search marketing best practices, from search-friendly store locators to data management at local search engines to review campaigns. Independent small business owners simply tend to be more engaged in their digital success than corporate managers.
- Restaurateurs in competitive markets tend to be much savvier about their digital marketing opportunities than those in less-competitive, typically rural markets.
- Engaged restaurateurs are more likely to pursue proactive reputation management campaigns (see bullet-point number two).
3. "Filtered reviews" ≠ "useless reviews"
I consider my friend Joanne Rollins to be a fairly typical small business owner. She runs a small frame shop with the help of a couple of employees in a residential neighborhood of NW Portland. She's not shy about sharing her ire with Yelp, not only around some of their shady sales practices, but especially about her customers' reviews getting filtered.
Trying to explain some of the criteria that cause a review to be filtered simply takes too long, and Joanne is easily frustrated by the fact that a faceless computer algorithm is preventing testimonials from 13+ human beings from persuading future customers to patronize her business. On the customer side, they're usually disappointed that they've wasted time writing comments that no one will ever see.
But all is not lost when a review is filtered! With permission from the customer, I encourage you to republish your filtered Yelp reviews on your own website. There's no risk of running afoul of any duplicate content issues, since search engines cannot fill out the CAPTCHA forms required to see filtered reviews.
You as the business owner get the advantage of a few (likely) keyword-rich testimonials, and your customers get the satisfaction in knowing that hundreds of future customers will use their feedback in making a purchase decision. Marking these up in schema.org format would be the icing on the cake.
4. "Filtered reviews" ≠ "reviews lost forever"
A review once-filtered does not necessarily mean a review filtered-for-alltime. There are steps that I believe will make their review more likely to be promoted from the filter onto your actual business page:
- Complete their personal Yelp profile, including photo and bio information.
- Download the Yelp app to their mobile device and sign in.
- Connect their Facebook account to their Yelp profile.
- Make friends with at least a handful of other Yelpers.
- Review at least 8-10 other businesses besides yours.
- Leave at least one review with each star rating (i.e. 1-, 2-, 3-, 4-, 5-).
For those customers who are super-frustrated by Yelp's filtering of their review or with whom you, as a business owner, have particularly a strong relationship, consider requesting that they undertake at least a couple of those tactics. I certainly don't guarantee their success, but it's worth a shot.
The reality of Yelp's review filter
As the infographic above demonstrates, Yelp's excitement over the citation from the NYAG as having the most aggressive filter underlines a fundamental business problem for the company that I've written about for years.
Yelp's fortunes are tied to their success in selling business owners advertising. Yet these same business owners:
- don't understand how the site works (at best)
- think that every Yelp salesperson is out to extort them (at worst)
Despite commendable efforts like their Small Business Advisory Council, Yelp clearly has a long way to go in educating these business owners. And they certainly have a long way to go with reining in rogue salespeople.
But the bigger issue is the consistent disconnect with their customers on the issue most important to their businesses--their guidelines for solicitation and display of reviews. Until they resolve that inherent conflict, I find it hard to see how they'll grow their revenues to the levels that Wall Street clearly expects.
David,
Thank you for the article. Incidentally, my business has been posting our "filtered" reviews on our site for over a year now. We elected to simply not allow Google to crawl that page for fear of duplicate content. As of today, we have 4 non-filtered Yelp reviews in our Denver operations and 81 filtered. Of those 81 filtered, 80 are 5 star. It is important to note that we have never contributed to a fraudulent review. It would be against our ethos in every possible way...
I wrote a blog post in August at ShatterBuggy.com entitled "Why Yelp is Bad" and it touches on many of your points. It is my believe that when you so impress a customer (positive or negative) as to encourage them to join Yelp just to review your business, that should be seen as quite powerful. Not the other way around.
B.
Benjamin,
Thanks for stopping by; sorry to hear about your situation. Looks like you are in the "typical business owner" boat I describe above, and perhaps even more extreme than "typical."
I've 'filtered' Yelp's response to the NYAG's statements... My filter is quite 'aggressive' when it comes to self serving misrepresentations from Yelp ;-)
I like the idea of 'republishing your filtered reviews' on your own website. Could there be any issue with a violation of Yelp's TOS?
Josh, thanks for the comment. Certainly, it's important to get permission from the customer before republishing their review on your own website. But Yelp very clearly states in its Terms of Service:
"As between you and Yelp, you own Your Content."
Meaning the customer is free to do what she wishes with Her Content.
Great suggestion on having the client publish the reviews on their own site but a word of caution on that. If they are re-published as in this case, then the client not only needs the customers permission but the review page needs to disclaim that they are published with permission on behalf of the customer.
Here's my experience. Yours may vary from this, but for my company:
But for our favorable reviews:
That's our experience. Yours may be different.
Yelp is trying to give public authentic reviews.
But they should explain business owners that the review filers are important to ensure the right review. They should do it soon else yelp reputation and people’s perception will change
Republishing filtered Yelp reviews on your own site is a phenomenal idea. Just wanted to let you know that filtered reviews are definitely still indexed, though. See for yourself.
David - Great article. Thanks for sharing your insight. Since Yelp is such a nightmare, what is the best alternative website for my customers to write a review?
Honestly, I get a headache when I read about all the complications and regulations that Yelp imposes on businesses and their patrons. Seriously - if you're not allowed to tell your customers that you're on Yelp, who is going to review you? People must either really really love you, or really really hate you.
Let's say that a customer loves you soooo much that they feel compelled to find you on Yelp. Once they write their shining review, there is a very high chance that it will get filtered out because (1) they gave you a 5-star rating, which is obviously "too good to be true" (2) they haven't reviewed 8-10 other businesses on Yelp (3) they haven't used the "perfect blend" of 1,2,3,4 and 5 star reviews for other businesses (4) they haven't fully filled out their Yelp profile "including photo and bio information" (5) they haven't used Yelp as a social network to make friends and (6) they don't want Yelp tinkering with their Facebook account.
Because you're not allowed to ask customers to review you on Yelp, most reviewers probably fall into one of two categories:
(1) People that are very active on Yelp anyway, who love to share their opinions with everybody.
(2) People that had a bad experience, who want to whine and complain, hoping to ruin your business.
I think that Yelp only works for businesses with thousands of patrons (like restaurants or hotels). If you have a very large pool of clientele, there is a reasonable chance that you'll get some "people that are very active on Yelp anyway" to review your business. But if your business works with a very small number of customers, you're unlikely to get reviews unless you ask for them, thus violating Yelp's guidelines.
Personally, my business is a software and web development shop. We have dozens of customers, not thousands. I have zero reviews on Yelp. Even if my customers decided to write a review on their own, it would probably get thrown out because of Yelp's absurd filtering criteria.
So... what is the best alternative website for reviews / citations?
Not sure about the alternatives, but yes it sometimes give bad experience when one's review (which is genuine) is filtered out.
Kevin,
We're in the same boat... we only deal with a few hundred customers
Hi SiteWizard,
Agreed, you don't want to encourage customers to review you on Yelp. It's a slippery slope when you're trying not to run afoul of their guidelines.
Mike Blumenthal has written some great articles about review source diversity & I'd encourage you to check out his blog for those. Citysearch is probably one of the better "horizontal" review sites because they syndicate far and wide. But vertical review sites (like Angie's List, Avvo, etc.) tend to be strongest in their given industries.
Interesting post as always David. I never thought that posts once filtered would show up, but apparently they do.
Hey David, I am a newbie to local SEO, just trying to bring our business up in local rankings. Tried Yelp, mixed results, and came to the conclusion that it if you can't beat 'em, join em. I looked in our community, Pinehurst, NC, for people who had posted more than 10 yelp reviews. I selected two or three who had lots of reviews, including positive reviews of galleries who represented our work. I opened a Yelp account, and began to aggressively review local businesses with whom I had experience, I filled out a complete profile, incuding a picture. Currently I have reviewed 75 or so local companies, in every ranking. I then reached out to the Yelp power users, especially those who had positive experiences at galleries where our work was represented, inviting them to come and visit us. I have received positive feedback from all invitations sent, and voluntary offers to review us once the visit has been accomplished. got a lot of compliments, helpful votes, funny votes, etc. I then linked my Facebook account to Yelp and invited my Facebook friends who had a Yelp account and had reviewed at least 4 other business to become friends. All accepted. So now I have all the components in place for Yelp credibility, claimed listing, lots of reviews in all categories, complete profile, and photo. It is my opinion that it is easier to convert an avid Yelper into a good customer than to get my good customers to become an avid Yelper. We are going to host an event for Yelp elites only, offer some specials, have a little wine and cheese, and see what happens. I will keep you informed.
Great insight, thanks for sharing, Jim! Looking forward to seeing your results.
Hi David! I think that republishing filtered Yelp reviews on our website is really a good idea. I will definitely try this. I just really hope that filtered reviews can still be indexed.
I hate to say it but with experience managing 2 different business pages on yelp and I have had very poor results. Basically the same thing happened in both cases 14 months apart.
It starts with the business getting a few good reviews, then a few more. NO manipulation, no fakes, hell the business owners can barely use a computer let alone write fake reviews. Monday morning happens, yelp calls about doing their CPC campaign, business owner is unsure if it is worth it and refers the Yelp exec. to me. I decide based on the low traffic it;'s not worth the $200-$300 per month. After a week of going back and forth with the Yelp exec. I say it's not for us right now but calls us back in a few months. 90 minutes later.... no seriously, 90 minutes later..... 80-100% of the business reviews are filtered. Hmm must be some type of anomaly in the Algo and that's how you get ants.
First off, excellent post, I am used to doing work with B2B clients, but even with them nowadays i find myself running into having to deal with negative reviews on Yelp, this article went a long way in helping me understand a lot of what seems like something I am going to need, even if it's not consumer focused going forward.
I think, as with any request for feedback such as on a site, offline venue specific paper review forms, consumer survey on receipts, at your dentist etc. you're going to get people from the most extreme ends of the spectrum. On one end you're going to get the people that absolutely love you, they can't wait to tell their friends, on social media, your online review sites, etc.
On the other end of the spectrum, you're going to get those who for whatever reason, had a bad experience and maybe even nothing you could do would of made them happy giving you a review that makes your business sound worse than the Federal Government during a shutdown. The people whose first reaction, mostly in anger, is to go on Yelp and give a scathing, maybe exaggerated account of the events that make it seem as if your company is incompetent, unapologetic and not to be trusted and warns everyone who will ,listen to stay away. This is the other end of the spectrum.
It just seems like the people who simply had an experience that was 'as expected', not 'disappointed', not 'impressed' isn't as represented in the online reviews as perhaps there are people who had that exact feeling after completing the sale and enjoying, using the product/service.
David, great article and well thought out. Aside from the allegations of extortion, (won't go into that) I think it's better at this point that sites like Yelp or over zealous in filtering reviews for obvious reasons.
Nice Post!
Hi David,
Again, helpful article as always
I was pondering what I'd do with a customer of mine who now is just sitting on one review in Yelp. It's a 1 star review and the customer was oddly dismissive of any attempts to resolve her issue as he has a 100% satisfaction guarantee. She has only 1 other 1 star review on her profile for another unrelated firm.
Yelp filtered out 2 other comments that were 5 star reviews. They seemed to be from infrequent participants as well. From what I understand, I shouldn't encourage him to have a page on his website where he points to his profiles on sites like Yelp, Google +, Angie's List etc. as this is against their TOS so I'll definitely keep this bookmarked for him and other clients.
Thanks!
Right or wrong, our solution is to include links to our various review profiles in an email. We follow up with each customer and send an email asking them to post an online review and give them three choices. Our thinking is that we can change the links to point to different sites as each profile fills up. Also the email will not be seen by a robot (with the exception of gmail's prying eyes).
We just (last week) started this strategy, so no results to report yet. If they filter all the review, I guess we could include detailed instructions in the email (post a profile pic, make a friend). We may also try linking them to yelps homepage and searching for our company by name rather than pointing them directly to our profile page.
Would greatly value David Mihm's input. Super timely post btw.
Thanks for clarifying aspects of the Yelp filter David. Diversification of reviews is a great strategy. For B2B, LinkedIn is a great place to get recommendations (as opposed to endorsements) and in my experience, there is no recommendation filter on LinkedIn. The other benefit of LinkedIn is that there is a good amount of detail about the person leaving the recommendation assuming their LinkedIn profile is complete.
For B2C, providing customers/clients a variety of review options is beneficial and serves at least 3 purposes:
1) Makes it more likely that a customer's review preferences are met (i.e. able to leave a review without having to create a new profile)
2) Provides businesses with diversity in terms of minimize risk of filtering minimizing the impact of loss of any one platform
3) Provides broader coverage for discovery and evaluation as potential customers use a variety of platforms to inform and influence their purchase decisions.
Google+ YP.com, Insiderpages.com, Manta and Yahoo! local are ones we have had some success with. For restaurants, sites like Urbanspoon are important. While in a different category, Facebook is important for word of mouth.
Relying too heavily on Yelp or any other single review site is a risky strategy when it comes to local reviews. Like investing in any assets, the key to long term success is diversify, diversify, diversify :)
Hey David, thanks for this information!
I was wondering how Yelp knows a business owner is asking for reviews. A business owner in Seattle asked me to look at his Yelp situation -- he had lots of great, filtered reviews and just a few unfiltered negative and positive reviews. He'd contacted Yelp and his Yelp rep encouraged him to include a badge and a link to Yelp on his homepage. However, he'd placed his Yelp link on a page soliciting reviews. I suspect that most of his customer reviews were submitted via that link.
I told him I thought that Yelp looks at the text surrounding the Yelp link and flags reviews from links with "please review us" and the like in neighboring text. Sound reasonable?
Sorry I missed you and Mike in NY earlier this week, but I'm looking forward to Local U either in Dallas or hopefully in Houston early next year.
Definitely sounds reasonable, Paul. I also suspect that if TOO high a percentage of a business's referring URLs that lead to reviews are from the business's own website, this may be a simpler way for Yelp to detect review solicitation (rather than trying to parse out surrounding text).
We will see you in Dallas in January!
Good point but that would be speculation at best on Yelp's part. It's entirely possible that the business owner does a fantastic job have getting clients on their web site.
Thank you David,
I have given up with Yelp. They have locked me out of my own account, will not even let me update It, do not respond to dozens of emails and messages I have sent them and only leave “sticky” reviews of fake people I have never spoken to or had the opportunity of servicing. It is clear that their bullying and strong arm / extortionist ways are not behind them. They do not return emails, phone calls, or requests to update my own business listing. Shameful IMHO
Russell Hartstein Fun Paw Care https://www.yelp.com/biz/fun-paw-care-miami-2
David,
Thank you for the article. Incidentally, my business has been posting our "filtered" reviews on our site for over a year now. We elected to simply not allow Google to crawl that page for fear of duplicate content. As of today, we have 4 non-filtered Yelp reviews in our Denver operations and 81 filtered. Of those 81 filtered, 80 are 5 star. It is important to note that we have never contributed to a fraudulent review. It would be against our ethos in every possible way...
I wrote a blog post in August at ShatterBuggy.com entitled "Why Yelp is Bad" and it touches on many of your points. It is my believe that when you so impress a customer (positive or negative) as to encourage them to join Yelp just to review your business, that should be seen as quite powerful. Not the other way around.
B.
David,
Thanks for tackling this hot topic!
Does Yelp send any type of warning emails about fake reviews on their account? I could see people start doing fake review for their competitors if they knew it would get them fined & remove legitimate reviews.
Step 1: Sneak into Competitor's headquarters
Step 2: Leave 10 amazing Yelp reviews
haha, let me know how that strategy goes for ya. I could see people competitors buying cheap fake reviews that would guarantee their competitor gets nailed.
Benjamin,
To my knowledge Yelp doesn't send any warning emails. One would think a "review disavowal" tool for business owners who'd claimed their Yelp pages, similar to Google's "link disavowal," would be a win for everyone.
Thanks David for the response. Hopefully they'll get something like that soon. With the penalties they are handing out I could see competitors start doing this.
why waist so much of everyons time ! Thanks for your help David
wat a great post
This topic continues to get beat to death! I'd like to point out something pretty obvious that I never see any one talking about and I firmly believe it's why many reviews get filtered, especially positive ones.
Yelp is fundamentally a social network and as such Yelp wants users who participate in the Yelp community, it's plain and simple!
Creating a profile and leaving one review does not make a one a Yelp community member. Yelp is looking for consumers who like to write reviews and who like/want to be a part of Yelp's community by participating in that community.
Straight from Yelps "About" page #8:
The software looks at dozens of different signals, including various measures of quality, reliability, and activity on Yelp.
I think the most overlooked part of that statement is "activity on Yelp"!
Yelp filtered out almost all, but not quite all, of our favorable reviews until one of their sales reps called us to sell us advertising. After I declined to advertise with them, *all* of our favorable reviews were filtered out. Pure and simple.