One of the challenges with feedback from individual customers is that it can often be unreliable and can actually understate or overstate your service delivery.

We have all seen anecdotal data at work in our lives. Let’s say you are looking for a new clothes dryer, and you decide to ask your neighbor for a recommendation. Your neighbor tells you that “I’ve had Brand X for seven years and never had a problem.”

The problem with your neighbor’s assistance is that your neighbor is one person who owns only one dryer, and there is a distinct possibility that your neighbor’s dryer might not represent the larger population of Brand X dryers.

And that is the problem of Uncle Jimmy.

Who is Uncle Jimmy?

We all have heard about someone’s Uncle Jimmy before.

Who, at some point in their life, has not heard some version of this statement:

My Uncle Jimmy smoked and drank every day of his life, and he lived to be 100.

It might have been Uncle Jimmy, Grandpa Frank, or Cousin Sally — but I bet you’ve heard (or said) something similar to this before.

Now Uncle Jimmy has a problem — well, a few problems — but one in particular that concerns organizational leaders. Uncle Jimmy represents the epitome of anecdotal evidence. He is a single data point that draws us towards a conclusion that might differ substantially from what the actual scientific data tells us.

Uncle Jimmy might have had a great genetics, he might have had low stress, or he might have had any number of factors that made him the exception to the rule.

The science is pretty clear: Smoking kills you. Excessive drinking kills you.

The data from Uncle Jimmy is misleading in another way. Uncle Jimmy might have lived to be 100, but that statement also does not take into account his quality of life. It is hard to imagine that he was a specimen of vibrant health after a lifetime of drinking and smoking. Modern medicine can combine with good genetics to keep certain people alive for a very long time.

The reality is that “Uncle Jimmy” is not typical and basing your decisions on how he lived can take you down a path that is very bad for you.

Anecdotal Customer Data Tells Stories…

…and the stories are probably as true as Uncle Jimmy’s.

Let’s return to our clothes dryer example. If you followed the isolated recommendation you received, you would be making your decision based on your neighbor’s experience with Brand X. Yet, Consumer Reports rates your neighbor’s model as average and Brand X as one of the lowest for reliability.

Your neighbor’s dryer recommendation is anecdotal data at its worst, because it actually tells a story that is not broadly true.

So how does this Uncle Jimmy scenario manifests itself in organizations? Most commonly, when isolated instances of customer feedback are allowed to affect organizational decisions.

Our human tendency is to draw rapid associations from isolated bits of evidence. (See Thursday’s post: Pavlov’s Customer Service Reps for more on this topic.) In logic, this is called the hasty generalization fallacy, taking an insufficient amount of evidence and using it to make a broad generalization.

For example, let’s say that a customer tells one of your team members that the new store music is really annoying. That team member mentions it to another team member on the same shift. Now, two days later, you are told that “the customers really hate the new music.”

If you can keep in mind the problem of Uncle Jimmy, then you will know to respond to feedback like this with detailed questions.

“How many complaints have you had Ted?”

“Just the one.”

“And you Jill, have you had anyone complain to you directly?

“No, I was just there when the lady complained to Ted.”

When you follow this exercise through for the other shifts, it turns out that the one customer was responsible for the feedback you received. One customer out of the approximately 700 that have been through your doors since you went live with the new music “hates” it. That’s less than half a percent.

Anecdotal Data Can Still Be Useful

Anecdotal data can be useful, but it must be understood for what it is. It’s primary use is directional — pointing you in the direction of further research. For instance, if you received a few more complaints about the music in a short period of time, then it might indicate to you that you need to look into the music further.

Past that, relying on anecdotal data in any other way must be done with a great deal of caution. Anecdotal feedback is story masquerading as data. It takes a lot of stories analyzed through a proper framework to equate to good data.

And such is the problem of Uncle Jimmy. Uncle Jimmy exists in a world by himself. He is a loner and a liar, and he likes to steer people in the wrong direction just for kicks.

If you are not careful, Uncle Jimmy can get your whole team hooked on cigarettes. Just don’t expect your organization to live to be 100.