Rating things is all the rage. I suspect that rating scales can influence ratings given and that there are other factors that influence the ratings a person is comfortable giving publicly. Except for the “thumbs up” idea (only occasionally paired with a “thumbs down” to go along with it), there’s been somewhat universal usage of a five-tier rating system, but with no universal definitions of the different ratings. And I’m not sure anyone pays close attention to the definitions anyway.
If you read reviews on (e.g.) Amazon and Yelp, you see ratings given purely as a number of stars, but you don’t see a definition of the ratings. If you post a review, only then are you given a definition of the ratings. The same things apply to Angie’s List, except letter grades are used instead of stars.
Sites seem to not want to try to mess with readers’ perceptions of what it means to be rated one star or five stars; they assume that people just “get it.” However, sites seem to want to help reviewers pick a number of stars for their reviews, as if the reviewers don’t just “get it.” This is odd and asymmetrical.
Here are the rating definitions on Amazon, seen only by reviewers (but accessible to anyone who tries):
||I love it
||I like it
||I don’t like it
||I hate it
With Amazon’s scale, you give stars even when you hate something (one star) or don’t like something (two stars). This is the nature of using a star-based system that covers the love-hate spectrum. How many reviews have you read where the reviewer said, “I’d give zero stars if I could”? Reviewers don’t want to give out a star to something they hated, even though that is following the definition. This perhaps shows that some raters don’t know what their rating is defined to mean, or perhaps they are more in-tune with the reader who might neither know (or care) nor have immediate access to how the ratings are defined.
What I like about this scale is its symmetry. If you hate something as much as you could love it, you give it one star. If you dislike something as much as you could like it, you give it two stars.
Here are the rating definitions on Angie’s List, seen only by reviewers (but accessible to any member who tries):
The A-B-C-D-F system feels the most meaningful to me. Perhaps it’s my experience of sixteen years of school in the U.S., but rating something A-B-C-D-F feels more meaningful than rating something one-to-five stars. An A is coveted. A B is still good, but no one wants to get one. A C is really not good and represents failure to many people. D means unacceptably bad but not a complete failure, whereas F means a complete failure. Unfortunately, this grading system is not internationally universal.
Here are the rating definitions on Yelp, seen only by reviewers (but accessible to anyone who tries):
||Woohoo! As good as it gets!
||Yay! I’m a fan.
||Meh. I’ve experienced better.
||Eek! Methinks not.
I really dislike these definitions. The difference between four and five stars is “Yay!” vs. “Woohoo!” This just doesn’t connect with me.
A-OK means better than okay. For a restaurant that I found perfectly enjoyable, an A-OK rating sounds perfectly fair and logical. But if I think “just OK” or a C grade, then this turns into a rather insulting rating.
2 stars: This is the “whatever” rating, with only a twinge of negativity. Does this map to “I don’t like it” or a D rating? Not in the slightest.
1 star: This is the only fully negative rating, but it still doesn’t feel as strong as “I hate it” or an F grade.
Many Yelp users register with their real names and pictures, and I think not being anonymous inhibits giving an honest opinion in some cases. This doesn’t apply to restaurants. Many restaurants in my area get hundreds of reviews, so anonymity comes from no one caring who wrote a specific review. Most if not all restaurants that are not terrible (i.e., staying in business) end up with 3.5 stars. My conclusion is that any restaurant that stays in business is liked by enough people to give it a decent rating. Thus, the rating summary for restaurants actually provides essentially no useful information. Every restaurant I’ve looked up in recent memory had about 3.5 stars, regardless of how good it actually was (in my snobbish opinion)—not that a C+ is a very good grade.
You can review anything/anyone on Yelp, and lack of anonymity comes into play for certain categories of reviews. For example, you can rate physicians. It seems that most one-star ratings for physicians are based on someone’s one and only one bad experience. How many Yelp users will publish a five-star rating of their long-term physician? I’d venture to say very few, because most reviewers are reviewing one meal in a restaurant and not the years they’ve been seeing a personal physician. For that matter, who would give their personal physician fewer than five stars and still want to face them during their next visit? Thus, physicians tend to have either no reviews or mostly negative reviews along the lines of, “Stay away!”
The case of the service provider (e.g., someone with a contractor’s license) can be interesting, and I am guilty of this: either I write a five-star review, or I don’t write a review. There’s often no gradation in the reviews. I don’t want to be the first to post less than a five-star review. If I feel the service provider was perhaps unethical or crooked, then I might write a negative review. But the service provider might pester you in return. I’m not just saying this; it happened to me the first week I posted on Yelp.
There are many classics of human nature wrapped up in this. If you had a one-on-one relationship with someone, perhaps starting with an estimate, followed by days or even weeks of working together, and you were unhappy with certain things along the way, society teaches you to always be polite and perhaps let your unhappiness fester under your skin. Now, you have the opportunity to write a negatively tinged review. Will you? Unlikely. If your Yelp persona is in fact yourself, then you’ll still feel the need to maintain being polite. Let’s face it: being polite means, for the most part, being dishonest.
But heaping praise on others, especially publicly, is something strongly encouraged by society.
With contractors, you will often see almost entirely five-star ratings. The value of the ratings, when they are all good, is really about the quantity of them. Knowing that someone was happy is valuable information. A single five-star rating (as the only rating) is not that valuable. Ten five-star ratings lets you know at least ten people were very happy, and that’s good. There were probably some who weren’t entirely happy, but that’s okay. There’s always the risk that things won’t work out perfectly. What I hope for in the reviews (and what I try to give) is plenty of detail.
I think it boils down to this: For people you interacted with just once and had a bad experience with, you are more willing to give them a bad review. For people you interacted with multiple times and had a less than stellar experience with, you will not want to rake them over the coals. Giving praise is easy, but giving criticism is hard, especially when you’re not anonymous.
The problem with averaging star ratings
Understanding online star ratings
Black Mirror, s03e01, “Nosedive”