The 5 Star Fallacy

The 5-Star Fallacy: How Uber Ruined the Single-point Rating System

Prior to the last, say, 3 years or so when searching for and assigning ratings to services or products online, there was a universal understanding of the scale, weight, and reliability of a 3-star vs. 5-star rating. Now, I feel, this is no longer true. And I blame Uber. Well, not Uber exclusively. Rather, I blame how we’ve placed reward on the expected (and many times, unreasonable and unrealistic) results, rather than using those results to inform, implement, and measure an improved outcome.

“If You Ain’t First, You’re Last”

In the modern comedic satire Talladega Nights: The Ballad of Ricky Bobby, Ricky’s father imparts some (rather flawed, yet motivating!) wisdom on his young son: “Ricky, if you ain’t first, you’re last”.  While this fueled Ricky to go all out in each race he pursued, it meant that – as a driver, teammate, parent, friend, and spouse – he left no opportunity for improvement. It was either win or crash.

In psychology this is referred to as ‘splitting’, where an individual is unable to recognize or acknowledge the variance of both good and not-so-good qualities in order to create a more complete, connected assessment. When rideshare company, Uber, came on to the market, there was a (mostly) unspoken understanding that – unless a driver had a 4.6 star average – they could be dropped (essentially, fired) from Uber as a driver. The following picture explaining the rating system from one driver pretty much spotlights the critical flaw with this policy:

Caroline O'Donovan / BuzzFeed News

Through basing hiring and firing on the notion of exceptional or nothing, as evidenced above, legendary is in the same category as a mediocre or decent experience. The act of simply completing the task one was asked to perform without causing physical or emotional harm is all that is needed to warrant 5 stars! OK is now the best. The driver even clearly states that 4-stars “does not mean average or above average”. If 5-stars indicates “I can improve” then how and when can you ever know if they have?

I have personally struggled with the Uber rating model. I can count on one hand the number of drivers and rides that were exceptional and worthy of 5 stars, where nothing could have made the drive more enjoyable: immaculate car, snacks, drinks, phone chargers, and music preferences offered. I feel a tinge of frustration anytime I must rate a driver at the end of the ride. I have even avoided using my phone for the request, leaving the rating to someone else’s conscience. In fact, I feel angry and as though I’m being strong-armed into awarding exceptionalism to someone who was not, due to the burden of being responsible for one’s livelihood and employment. They may have been a good and pleasant driver, but they did not provide the same level of service as those who went the extra mile (pardon the pun). If everyone must be considered exceptional in order to not get fired, then no one is. In addition, if an OK (3 or 4 star) driver continually receives 5-star ratings, how will he or she ever know there are opportunities for improvement?

Michelin has a 3-star rating…and it’s not awarded for mediocrity. Chefs work for years – some even a lifetime – to have the opportunity to be bestowed the coveted top tier status. For anyone not a foodie or fan of “exceptional cuisine that is worth a special journey”, they may think a 3 Star-rated restaurant (based on Uber’s scale, above) “sucks so bad I never want to [eat there]”.

Quid Pro No

Another way in which the value and inaccuracy of ratings and loyalty deceives is through a new social media marketing scheme I like to call “baucus-style marketing”. It’s when, in exchange for a perfect rating, ‘Like’, ‘Follow’, or ’Share’ the person receives some sort of reward or kickback. Maybe it’s a discount on a service, an entry into a contest, or – in the business world – an extra point on their performance assessment.

While it’s true that you should try and reward the behavior you want to see, doing so through manipulation and bribery is a not-so-great way to go about it. Through this technique, you have businesses who have tons of likes, a mountain of 5-star ratings, and with no revenue or terrible service. Employees or clients appear to be engaged, but have little real interest in your business. From an HR perspective, how can you decipher between the A players and the 4th string?

How Can We Resist?

Despite our current state of misinformation around ratings, I feel there are opportunities we can begin to adopt and encourage (or even demand) within our own lives, businesses, and activities we patronize.

  1. Accept that “Average” is not a 4-letter word
    • It’s time we reset our single-point rating system back to what it used to mean:
      • 1 = Poor
      • 2 = Fair
      • 3 = Average/Good (minimum acceptable satisfaction)
      • 4 = Above Average/Better (a slightly better experience or outcome than expected)
      • 5 = Exceptional/Best (a memorable experience or outcome, rarely encountered)
    • Average is not poor or unacceptable. It’s simply the baseline for ‘good’.
  2. Get clear on what you’re rating
    • As a business, get clear on what you’re trying to measure and strive to educate your customers and clients on what and how your rating system is used to continually assess and adapt your business operations and people.
    • I encourage you to place more value (from a growth vs. pass/fail perspective) on a 5-star rating. Do you want your business to be mediocre, or do you want to understand how and where you can grow and improve? If you aren’t placing high value on the quality of your service, people, products, or offerings, then you are not truly invested in delivering an ‘exceptional’ experience. However, don’t fall into the ‘Uber’ way and consider less than stellar ratings as a means for disciplinary action or termination. While sometimes a hard pill to swallow, the reward for ‘taking your lumps’ and developing action plans for improvement – be it for an individual, a department, or a process – will undoubtedly lead to more positive ratings.
    • Also, as a customer or client, rather than treating every rating like that of (in my opinion) the worst offenders, inquire what the ratings mean for the business(es) you are collaborating with and patronizing. This way, you can ensure you are providing them with a rating that aligns with the measurement that is important to them.
  3. Don’t take the bait
    • Stop participating in or persisting baucus-style marketing. As a business, I know it can be tempting to create an incentive-based campaign for likes or ratings. It’s just as tempting for the customer who may be participating to like a page you either don’t love or have never heard of, or give a perfect rating to a company or person who was just ok in order to receive a ‘prize’. But, think about how you are contributing to and perpetuating the fallacy of exceptionalism to others? Would you want a friend or loved one going to a restaurant or buying a product that was just ok? Then why make it seem amazing to others?
    • As Facebook’s latest algorithm changes indicated, ‘engagement’ has a renewed definition and includes real feedback and meaningful communication. As a business, simply encouraging and promoting likes will no longer improve your social media presence or saturation.
  4. Look for new ways to measure satisfaction
    • There are two schools of thought when it comes to improving how we measure satisfaction
      • Keep it simple
      • Be more specific and targeted 
    • I will leave it up to you to decide which path is best for your organization; however, I feel both are valuable tools that may be used to aide in creating a clear picture of your operations.
    • Using a simple Net Promoter-based rating may be good for periodic check-ins with your employees, clients, talent, and partners. It’s quick, it’s easy, and can help spot sudden variations, dips, and spikes for quick resolution. However, a single rating method without the ability for the user to provide qualitative feedback and context diminishes its effectiveness. Either through the technology or through a follow-up interaction, the evaluator should be able to provide more around the ‘why’ of their response.
    • Erecruit Tip: Using Erecruit’s Smart Form technology, coupled with its workflow engine can ensure these assessments are delivered on a regularly-scheduled basis and standardized based on your staffing firm’s quality measurements.
    • Using a performance platform that can capture targeted ratings for multiple performance indicators such as quality, competency, attitude, professionalism, and performance at important milestones within a project or relationship, can help both businesses and their clients identify and assess specific areas of improvement on a repeated and on-going basis, rather than simply waiting until an assignment has concluded.
    • Erecruit Recommended Best Practice: Send out an evaluation following completion of a candidate’s onboarding. Consider a few touchpoints including proceeding initial screening, pre-hire onboarding, and assignment confirmation and start. Throughout the assignment or project period, also send quick check-in assessments to gauge their level of satisfaction with the assignment, their Recruiter, the Client, and any other meaningful measurements you wish to learn more about.
  5. Reward improvement
    • Don’t simply reward exceptional outcomes but recognize and reward when improvement occurs. Creating value around upticks in a rating’s result versus simply expecting the best and punishing those that fall short will encourage continual drive and desire for improvement instead of discouragement (and even, abandonment).
    • There’s a joke amongst my friends that my rating scale (1-10) ends at 9, because hardly anything (even my husband or myself) is worthy of a 10. But, you know what? That’s ok. It doesn’t mean I am continually disappointed, rather it’s that I recognize there are always opportunities for improvement, in most any product, person, business, or situation. And I feel those instances where it’s been amazing, I want to be able to recognize it, authentically. Heck, even Reese Bobby realized the error of his advice, acknowledging that 2nd or 3rd place is still commendable.

If he can do it, then I am pretty sure the rest of us can. Let’s embrace average, learn from the feedback, and continue to strive to be exceptional, knowing that continual improvement is the real measure of our success.

For more information about Erecruit’s survey and performance measurement features, as well as some of the companies we’ve partnered with to promote quality measurements and assessment, contact your Erecruit Project or Account Manager, or sales@erecruit.com

For tips on creating and communicating effective surveys and evaluations, check out Inavero’s Satisfaction Survey Checklist for Staffing Firms.


Amy Yackowski is the Director of Healthcare Best Practices for Erecruit. Amy is responsible for helping Erecruit healthcare customers develop their framework, analyze their business processes and optimize their use, effectiveness and efficiency of the Erecruit solutions to boost their profitability. Join her in conversation on TwitterLinkedIn and ayackowski@erecruit.com.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *