|
You cannot discuss online rating systems without mentioning Amazon.com.
As the largest online retailer, Amazon has always been at the forefront of online shopping and it’s comment system is one of the most, if not the most, sophisticated out there. Amazon has introduced numerous innovations over the years to add credibility to its rating system:
All of these innovations have helped add a little structure and order into the freewheeling, anarchic world of online ratings.
Are there many frustrations greater than trying to buy a heavily rated product online? Truly.
You’re almost better off if the product in which you are interested has 10-12 reviews instead of 200. The more reviews the greater the chance that every concern you have about the product will be realized.
Worried that style of washer vibrates too much? Or that the edges on the level won’t be smooth? Someone will have had the problem and written about it.
The sheer volume of opinions and experiences is challenging enough — what is more difficult and frustrating is when reviewers rank the product based on the experience of the purchase and not the product itself.
Do you really want your comparison of high-end digital cameras you are performing muddied by one-star ratings that come from incorrectly delivered packages and people who did not read the product description carefully enough?
I know its a digital camera, but I really wish it took film too. I have three cases of Fuji Film sitting in my office that I need to use. 1 Star.
And while little can be done to mitigate the opinions of consumers who lack perspective, there is an even greater challenge with comments that are legitimate but are not directly related to product quality. A few key complaints can consistently be found in the “poor” reviews on Amazon, and it is these complaints that weaken the effectiveness of the review system.
To begin, online ratings systems will always be the epitome of imperfect data. Competitive spammers, perspectiveless consumers, and just good old fashioned poor reviewing will always make taking actionable intelligence away from online reviews as much art as science.
However, all systems are imperfect, and that does not mean we should ever stop trying to perfect them.
In my humble opinion, Amazon could improve its rating system by offering two separate ratings scores — a Pure Product Score and a Blended Score.
The goal would be to have a Pure Product score which, as much as possible, is not affected by the tangential items that do not have to do with core product performance.
This could be accomplished easily by adding a simple, mandatory drop down box prior to review submission. Amazon could ask a question such as…
Does your review include commentary on any of the following:
Once the key questions are established, the ratings would be simple. If the customer chooses Yes from the drop-down menu, the review only goes into the Blended Score. If the customer, chooses NO on the drop-down, the score would go to both the Pure Product Score and the Blended Score.
Obviously, I’m winging this. Amazon has some of the best quants in the world mining its data and improving its systems — so they can figure out which questions would really move the needle on separating pure product reviews from muddied reviews.
Regardless of the detailed execution, this separation of scores would help prospective purchasers have a better sense of consumer opinions on the quality of the product they are interested in, with less tangential noise cluttering their decision making.
Amazon’s rating system is one of the most innovative and thorough in the world, but it faces the same quandary that has plagued users of customer feedback since the dawn of surveys — translating anecdotal evidence and opinion into quantifiable and useful data.
To me, attempting to maximize the purity of product scores is a logical next step in that evolutionary process.
Comments have been closed on this post.
© 2011-2023 CTS Service Solutions, LLC.
All rights reserved.
Legal Information | Privacy Policy
How to Cite this Site
Personally, I like Klout’s rating system because it gives me free stuff like you did. My latest was a Walkman; now if I can just get them up to some TaylorMade golf clubs we’ll be cookin’ with gas.
I do agree that it could be tweaked because there could be many, many more perfectly satisfied customers who don’t take the time to rate something. Therefore it could be skewed but the unsatisfied few who are probably the people who always send their food back too.
Just sayin’…
The good news with Amazon is that there is such a culture of reviewing that you tend to get a more balanced set of reviews than say, the local coffee shop. Often with those stores, you get the people that love you and the people that hate you, but not the many people who like you a lot.
I like the idea, but it does rely on users being both truthful about the nature of their review, and not easily confused by an extra option.
To go one step further, I’d like to see the ability for other users to flag another person’s review as not being a “pure” review.
Perhaps a slightly simpler approach would be if the review required you to specify what you’re actually commenting on – e.g. product, delivery, customer service – and then not only would we have clearer aggregate scores, we’d also be able to filter out the non-products reviews themselves (rather than just the scores) when reading people’s comments.
Hey Ben, I think the idea of other people flagging reviews is interesting. Could be ripe for malfeasance, but maybe there are some controls that could be put in.
Not sure how what you are suggesting is simpler really; it actually seems more detailed (but I might be misunderstanding it). My thought was that if you aggregated all of the “non-product” scores as a group, it would at least get us closer to a pure product score. To your first point, I would be concerned that even that yes/no single option would confuse some.
Thanks so much for commenting! Good stuff.
Pingback: Support Hangout Episode #1
Pingback: Monthly Mash and Self Service
Pingback: Customers Lying Online: An Interesting New Study