Many of us with social science backgrounds learned the cautionary tale of the Heisenberg Effect in nuclear physics. As physicists find it impossible to observe certain subatomic phenomena without altering them, so the very act of measuring human behavior or attitudes may alter them as well. Another of the teachings of the social sciences which we ignore at our peril is to look for the unintended consequences, or latent effects, of what we do.
I submit that the customer satisfaction industry, both naively and cynically, often ignores these basic principles. In so doing, the credibility of genuine marketing research is undermined. The well of public trust is being poisoned slowly.
Specifically, I believe that certain customer satisfaction measurement practices substantially alter the phenomena we seek to describe in undesirable ways. Whenever the raters are in personal contact with "ratees" whose fortunes are affected by customer satisfaction research, we can only expect the latter to manipulate the outcomes. We in the professional research community cannot escape blame. We aid and abet the very abuses of which I write.
Who among us has not had experiences such as these?
Even though my personal sample of one should be suspect, what we know of elementary behavioral psychology should make us wary all the same. When tens of thousands of people are directly punished and rewarded based on numbers generated from customer satisfaction surveys, it is perfectly natural that they will modify their behavior to get better numbers. That behavior includes blatant efforts to persuade customers to rate them better.
This fact of life is curiously missing from all theories of service quality and customer satisfaction, not to mention from learned discussions of modeling techniques, the number of scale points to use, data reduction procedures, and questionnaire construction. Is the entire customer satisfaction industry blind to the unintended consequences of its measurement procedures and to the Heisenberg Effect?
I am surprised that methodologically sophisticated researchers and marketers who are close to the customer satisfaction industry have not begun to sound an alarm over validity. Crudely put, a valid measurement is one which accurately represents what it is you think you are measuring. How much of the variance in customer satisfaction scores are due to true differences in customer satifaction--and how much is attributable to the skill of service providers in manipulating their own ratings? If a dealership has gone from 89 percent "fives" to 91 percent, what has changed--its service quality or its sales staff's ability to beg those precious "fives?"
If those being rated are not in a position to manipulate raters, then the issue disappears. Customer satisfaction scores will mean more or less what they are supposed to. Where, however, the ratee can personally pressure the rater, I would regard the eventual ratings data as bogus.
It disturbs me even more that ostensibly professional marketing researchers are enablers of the collection of such compromised data. We may not go so far as to wear white laboratory coats, but it is tempting to play scientist in an antiseptic office, writing questionnaires, processing bytes of data, and generating dispassionate, objective reports, all on our computer (I enjoy this role as much as anyone). However, we should never forget that it is sentient, willful human beings who participate in customer satisfaction surveys.
How can we purport to understand the attitudes and behaviors of people as consumers if we are blind to their attitudes and behaviors as respondents? Either our contribution to business decisions is even more flawed than our severest critics allege, or our hypocrisy borders on the unethical.
Furthermore, what are people to think about the credibility and seriousness of any survey research? On-line "polls" are bad enough. I just see more toxins being poured into the well of consumer good will upon which most of us researchers (and ultimately our clients) rely.
The rationale for measuring and acting on customer satisfaction is admirable--listen to the consumer, adjust the marketing mix accordingly, and reward those service/product providers who do the best job. Who could argue with that?
What we can argue with are execution systems which are vulnerable to easy manipulation. They have been created jointly by those who know little (or worse, care little) about human behavior, and those who should know better but have chosen to look the other way.
There is a market for customer satisfaction measurement services. Aren't we obliged to meet this need, afterall? There is a demand by apparel firms for inexpensive manufacture of clothing, and illegal sweatshops right here in the U.S. of A. exist to meet that demand. Does that justify their practices? There are arguments favoring sweatshops, but I think otherwise.
Similarly, I would eagerly listen to arguments that justify reward systems based on compromisable data and manipulable respondents. Whether to laugh or cry could be the tougher decision.
What is
to be done, ultimately, is to use our intelligence. Whether the abusers of customer
satisfaction research are outright unethical is a matter of judgment. That they are stupid
is a matter of fact. We can hope that Darwinian forces lead them to perish. The risk is
that legitimate marketing researchers will suffer and perish along with them.
We all drink from the same well of consumer trust.