Matt Freeman DNP, MPH
“10 Out of 10”
When I purchased a car some years ago, I remember it as a favorable experience. The saleswoman was organized, cheerful, and I was given a generous discount. But the experience left a bitter taste in my mouth. As I left, she said, “You will receive a survey soon. Be sure to give me ’10 out of 10′ on everything or I will lose my job.”
It seemed a little hyperbolic. How could one survey wreck this woman’s livelihood?
A Vice President for the particular car manufacturer had gone to my school, and I decided to send him an email. He called me within the hour. “That’s not how this is supposed to work,” he said. “If we only got ’10 out of 10′ on everything, why would we bother asking?”
Aside from the expectation that all customers will give a perfect score, Likert scales are easily misinterpreted. One can picture “metrics” displayed in the break room of the auto dealership: “Our average satisfaction went up from a 9.0 to a 9.25 last month!”
This is miscalculation. One cannot take a mean or average score of Likert survey responses. Using means or averages makes an assumption of equidistance: the difference between a “7” and an “8” is assumed to be the same as the difference between an “8” and a “9.” There are statistical methods for analyzing Likert data, but these are often absent from social science research, so one would be unlikely to see robust statistical models at an auto dealership.
The greater question is, “What are you going to do with the information?” Let us imagine that the auto dealership was earning a “2” from most of its clientele. What can the sales personnel do to correct this? Be nicer? Offer candy? Flirt? The salespeople arguably have little control over the ratings that their customers submit. Perhaps there is a red flag if one salesperson uniformly gets lower scores than everyone else, but one would assume that there would be other indications of a problem, such as a poor sales record.
The “10 out of 10” Expectation is Dangerous in Healthcare
“Every patient after every visit.” It is a line I have heard from multiple directors of primary care services. “Everyone gets a survey.” The goal seems to be to a mass of data collection.
It is hard to see how this information would not be skewed. At one HMO where I was a patient, I was asked to check survey boxes before I had even met with the clinician who was to take care of me. Only the most daring patient would write anything negative while his or her doctor, nurse, or other provider was sitting right there.
One can see how the advance survey might work in a healthcare provider’s interest. A patient might be about to receive bad news: “Here, give me a 10 out of 10, and then I will tell you about your poor prognosis, or how I want you to quit smoking, why you need to exercise more, or whatever else you might not want to hear.”
In clinics where I have worked, patients were given slips of paper with Likert scores immediately after their visits. Although anonymous, the results were typically pointless: “20 people gave us ‘5 out of 5’ this week.” It provides a nice pat on the back, a reassurance to management that our patients appeared to be happy, and we could conveniently say, “We got the same great scores a year ago.”
Even those surveys that offered the option for narrative responses, these were not constructive. Patients might write, “Everything was fine,” or “I like nurse Beth.” Nice to know that Beth is appreciated, but there is nothing to do with this information.
Satisfaction and Wellness Can Be Inversely Related
In a study of 52,000 patients, researchers from the University of California Davis identified that patients with the highest satisfaction scores had a mortality risk 26 percent higher than less satisfied patients. The most satisfied patients were less likely to have emergency department visits, but were more likely to end up hospitalized, have greater healthcare costs, and be on more prescription medications.
There are multiple theories as to why the most satisfied patients are dying sooner. Researchers particularly identified prescription drug expenditures as an indication that patient expectation guides clinician behavior. A patient expects a certain medication, and is satisfied if the prescriber orders it without respect to cost, risk, or medical necessity.
The satisfied patient may also be hearing only what he or she wants to hear. In order to boost survey scores, providers may be ignoring more difficult conversations about adherence to medication regimes, lifestyle issues like weight or smoking, or similar concerns that could upset a patient.
One physician explained to me that he and his colleagues abbreviated their physical examinations because the momentary immodesty or embarrassment lowered their patient satisfaction numbers. It is rather like saying, “We stopped giving tetanus vaccines because patients might complain that their arms became sore.”
Pay for Performance
Two years ago, the Center for Medicare and Medicaid Services (CMS) began including patient satisfaction into hospital reimbursement. The stakes are high: about $1 billion in annual hospital payments is based upon responses to a 27-question patient satisfaction survey. The survey is not the only “pay for performance” measure used; hospitals and their staff are also evaluated on their adherence to standards of care, and other presumably measurable elements of patient care. The patient satisfaction survey accounts for 30 percent of the “pay for performance” payments.
Hospitals have struggled because patient satisfaction is unpredictable. Furthermore, patients are not admitted to the hospital for a positive experience. “Do you think it is a great experience when I tell you that you have stage-four cancer and you may be dead in three months?” explained a chief nursing officer.
Another nurse executive recalled a patient who was fortunate to survive a stroke but complained that the meals in the hospital were too cold. Surviving a life-threatening illness and receiving high-quality care for a stroke still cost the hospital a pay cut because the food was not to the patient’s satisfaction.
The circumstances can be far more hazardous than just lukewarm hospital food. A nurse questioned a South Carolina emergency department physician when he ordered hydromorphone (Dilaudid) for a woman with a toothache. Hydromorphone is a powerful narcotic that is actually used in executions by lethal injection. The physician explained that his patient satisfaction scores had dropped in the past month, so he was making any effort to please patients, even if it was a bizarre choice of an unnecessary and potentially hazardous medication.
A family practice physician explained to me that he prescribes codeine cough syrup to every patient with a cough “because they enjoy it.” Although not as potent or as dangerous as hydromorphone, codeine carries many risks, and should only be prescribed if the patient needs it, not for a good time.
A Cheating Culture
A hospital executive explained to me that nursing staff were calling recently discharged patients to “coach” them on patient satisfaction surveys. Although the mechanism was unclear, there were rumors that the hospital staff found a method to ensure that the most problematic patients never receive the survey. When so much money is at stake, it is unsurprising that healthcare facilities would turn to dishonesty to manipulate survey results.
The manipulation of survey data is not unlike the scandal-fraught “pay for performance” efforts in public schools. Michelle Rhee introduced an elaborate pay for performance strategy when she became Chancellor of the District of Columbia Public Schools. Rhee had grand displays of $8,000 to $10,000 checks given to teachers and administrators when their students’ scores increased on standardized tests. Journalists from USA Today identified that teachers were “correcting” their students’ test responses. Once the District enacted a security policy that prevented tampering with test responses, the students’ test scores plummeted.
Michelle Rhee
The DCPS scandal was one of many. In 2013, Beverly Hall, Superintendent of the Atlanta schools, was indicted in a similar test manipulation scheme. When the financial stakes are so high, student achievement and ethics are cast aside. Dr. Hall herself received $500,000 in performance bonuses. She was described as a leader who, “allowed cheating—at all levels—to go unchecked for years.”
“Satisfaction” in the Healthcare Context
Aside from food served at the right temperature, how can healthcare providers and facilities ensure high satisfaction ratings and thus higher pay?
The first problem is that patients are usually sick! No relationship is going to feel particularly great if it is in the setting of an illness, needles, surgeries, tests, anxiety, and so forth. Furthermore, people who struggle with psychiatric disorders that affect interpersonal relationships are over-represented in primary care clinics. Studies have estimated an 18 to 26 percent prevalence of borderline personality disorder at a primary care clinics. That seems like an overstatement, but one could safely argue that a variety of psychiatric conditions are over-represented in those seeking primary health care.
If one is looking for favorable patient satisfaction survey results, looking to those who do not feel well or those with personality disorders would not be good choices.
In some instances, satisfaction survey tools and expectations are not designed by anyone with a healthcare background. In one practice where I worked, there was a “zero tolerance” policy for patient complaints. This is absurd. If one is ill, perspectives can be blurred: anxiety and depression can be exacerbated, patients and their families may seek to blame someone for an illness. Above all, the costs of healthcare can trigger complaints. I covered my own employees by “accidentally” failing to mention complaints. These were almost never substantive, and I often did not even bother telling the physician, psychologist, or nurse that anyone had bothered to complain. There was nothing to gain from the complaint other than anxiety and self-doubt.
I supervised one physician who was often the subject of complaints. Patients felt that he was not warm or engaging enough. He had a cerebral, introverted, and thoughtful approach. His medical judgment was sound, and he had a fantastic sense of humor. He had top-ranked credentials, and offered meaningful insight when we worked together as a team. What was I going to do with the complaints? Turn to him and say, “Change your personality. Watch this video about how to be more ebullient or I will cut your pay.” I never said a word to him.
Anyone who has worked as a healthcare provider or in healthcare management recognizes that “10 out of 10” from every patient is an unreasonable expectation.
“It Was a Pleasure to Participate in Your Care Today”
My Israeli colleagues were teary-eyed with laughter when they watched American instructional videos about how to improve their relationships with patients. The videos seemed to imply that one had endless time during visits, and that “canned” statements replaced authenticity. They were must amused by running consent narrative that was deemed to be satisfaction enhancing. “I would now going to look into you ear, is that okay with you Mrs. Johnson?” It is hard to imagine that Mrs. Johnson really cared that much; she probably just wanted her earache to go away.
A friend at a large university medical center is required to conclude every visit with, “It was a pleasure participating in your care today.”
Instructional videos, workshops, and guides designed to elicit higher patient satisfaction omit variation among clinicians as well as a critical force in the provider/patient relationship: authenticity. It seems self-explanatory that patients would rather converse with a real person rather than someone using pre-programmed speech and phrases.
Asking permission to look in someone’s ear or the odd expression of “pleasure” in participating in a patient’s care overlook the more critical role in better patient care: shared goals. One would assume that the most satisfied patients have their needs met. A patient may need to just talk, may just need pain control, or may need reassurance that his or her symptoms will improve with time. A savvy clinician seeks to establish to make shared goals and expectations.
I have worked with adolescents and young adults for 15 years, and I have seen thousands of patients for pre-participation examinations for athletics. My usual line is, “Are you the kind of patient who would like a ‘play by play’ explanation of what I’m doing, or have you done this a lot before, and would you prefer me to just ‘get it over with?” Patients almost invariably choose the latter. In other words, the patient’s goal is to just get out of there with their paperwork signed. That scenario is never presented in patient satisfaction training modules.
Weighing Medical Judgment, Ethics, and Scores
Sometimes the signature is not an option. Saying “no” is a part of medicine. What if the teenager or young adult does not meet the medical requirements for the particular sport? You can forget customer satisfaction. The patient (and probably his or her parents) just wanted a signature regardless of the sound reasons to be concerned about the patient’s health. Even if the answer is a request for prior records, a chance to talk with another healthcare provider, or an additional test, the visit is catastrophic from a satisfaction standpoint. Cheerleading practice starts this afternoon, and the doctor just told the cheerleader that she has to wait until the x-ray results are back because her wrist appears broken. There will not be a “10 out of 10” for the doctor that day, and he or she may face a pay cut because of it.
In a similar example, patients with sleep apnea are required to undergo a commercial driving license examination every year instead of every two years. If a healthcare provider wanted higher satisfaction scores, he or she would be wise to ignore the Federal Motor Carrier Safety Administration guidance and issue a two-year license. But this action puts patient expectation and satisfaction a priority over the safety of the patient and the public.
Like a teacher in Washington or Atlanta shortchanging students’ educations in exchange for financial incentives, a doctor can easily be lured by the threat of a poor survey response. Maybe he or she will “accidentally” overlook a broken wrist or sleep apnea, or just not examine the patient’s wrists or ask about sleep apnea. Then everyone can be happy about the outcome of the visit. Attention to medical ethics and the long-term consequences to individual and public health do not provide cash incentives.
Patients may ask for medications that may harm them, tests they do not need, approval to continue health-compromising behavior, unlimited access to their providers, and other unrealistic expectations. Of course one should say “no” with a combination of professionalism and problem solving. But the answer is still “no,” regardless of how gently and sensitively the message is conveyed.
Online Reviews
Satisfaction surveys impact reimbursement directly. Online reviews can prevent patients from coming in the first place, equally affecting a clinician’s income livelihood.
Yelp.com is the leader in online reviews. As with all of its reviews, Yelp does not verify if a patient was even a patient at all. Anyone can write a review any time. Restaurant owners bemoan Yelp reviews in which customers complain about the salmon but the restaurant does not even serve salmon. The same applies in healthcare: a Yelp reviewer does not even have to have met the doctor better yet visited as a patient.
In my case, Yelp was an invaluable resource. Without any money paid to Yelp, I ended up getting ranked “#1 Best Doctor” in my city. Patients flooded in. But it was not really fair. I was happy to have the business, but the Yelp reviews had nothing to do with my clinical acumen, education, or other abilities. In fact, subsequent practices consulted my Yelp reviews when they made the decisions to hire me.
Some cases were baffling: one woman wrote about a negative experience with me but continued to see me as a patient. Although I did everything possible to remain objective, I desperately wanted to say, “You defamed me in a permanent, public fashion, and now you want me to treat you?” I quietly ignored the review since Yelp is theoretically anonymous. This also exemplifies the over-representation of borderline personality traits or borderline personality disorder in primary care (borderline patients tend to follow the pattern of, “I hate you; don’t leave me.”)
Yelp could have easily broken my practice, income, and future employment. A handful of negative reviews, and five stars drop to four. Yelp and other online review sites have proliferated, so one can only hope that favorable reviews outnumber the duds.
Empowerment versus Manipulation
Patients who are aware of the financial implications of an unfavorable survey or Yelp review can influence the objectivity of their care, even if it is to their own detriment.
A friend told me of a patient who threatened his primary care provider with a negative online review. Consequently the practice overlooked the patient’s unpaid bills and withheld addressing his poor adherence to medications and markedly unhealthy lifestyle. He was to receive large discounts and only good news about his health.
Another former colleague faced a woman who snapped, “I’m going on line and telling everyone about you” because she did not receive the antibiotic she thought she deserved.
In the expanding dictionary of medical slang, the most frightening patients are referred to as “bcc.” This refers to the email selection of “blind carbon copy,” a patient who says nothing to his or her treating provider but submits a negative survey response or derogatory online review.
A friend outside of the medical profession asked me if I ever felt badgered in to ordering an inappropriate test or prescribing the wrong medication due to fears of a negative survey or review. “Absolutely!” I replied. He was shocked. “The customer is always right, even when the customer is a patient and you are trying to keep him healthy.”
Patient satisfaction is not to be confused with patient empowerment. The empowered patient is an active participant in his or her care, knowledgeable about his or her health status, assertive, asking questions, and willing to request a second opinion. This is reasonable and helpful.
My least favorite patients say, “Whatever you say.” I would much rather have a patient who is willing to say, “I was reading about this before I came in.” That way, I know what might be worrying the patient, what tests or medications the patient expects, and might shed light on a diagnosis or treatment that I had not considered.
The empowered patient is not threatening or manipulative like a “bcc.” He or she is honest and informed, and expectations are articulated. Above all, an empowered patient feels safe asserting his or knowledge, needs, concerns, or even misgivings. An empowered patient is not necessarily going to balk if a doctor contradicts what the patient anticipates.
The manipulative patient may or may not voice his or her expectations, unreasonable expectations, a personality disorder, or any other confluence of factors that can lead to dissatisfaction. This dissatisfaction can result in a form of “acting out:” the patient can argue that he or she has leverage in the form of surveys and reviews.
Moving Forward
Reviews and surveys have become embedded in American culture, and they are unlikely to go anywhere. But there are a few steps that could help make better use of the data collected:
- Satisfaction scores should never be tied to compensation. This leaves far too much room for cheating, and it can produce a culture in which patients are given inappropriate or even dangerous medications, tests, procedures, and advice.
- “Every patient, every time” collects an extraordinary amount of data, but it is unclear about how this can be used to improve patient experience. As with any survey tool, the first question to ask is, “How are we going to act on the results after we analyze the data?”
- “10 out of 10” from every patient is unreasonable, and is incompatible with safe, effective healthcare. As the auto manufacturer vice president put it, “What is the point of asking if you get a perfect 10 every time?”
- Any use of Likert scores should employ sound statistical models.
- Canned phrases like, “It was a pleasure participating in your care” can be demeaning to patients. These really just indicate that the healthcare provider went through a training course, and that he or she is likely subject to evaluation. Authentic, genuine, honest, and respectful communication should be the expectation.
- Healthcare providers and their employers have to remember that Yelp and similar online review sites are not a reliable measure of provider performance. In fact, healthcare facilities and providers should take the lead by educating the public on how online reviews maybe skewed. I write this knowing that Yelp was invaluable to me as I built a practice.
- Except in emergencies, patients overwhelmingly have a choice in providers and facilities. If a patient is dissatisfied, he or she would be wise to just seek care elsewhere. Just as if one did not like the steak served at a restaurant, it seems wiser to go somewhere else next time rather than berate the chef in a satisfaction survey.
References
Adamy J. US Ties Hospital Payments to Making Patients Happy. The Wall Street Journal. 14 October 2012. http://www.wsj.com/articles/SB10000872396390443890304578010264156073132. Retrieved 11 December 2014.
Falkenberg K. Why Raiting Your Doctor Is Bad For Your Health. Forbes. 2 January 2013. http://www.forbes.com/sites/kaifalkenberg/2013/01/02/why-rating-your-doctor-is-bad-for-your-health/. Retrieved 11 December 2014.
Fausset R. Central Figure in the Atlanta Schools Cheating Scandal Dies. The New York Times. 2 March 2015. http://www.nytimes.com/2015/03/03/us/central-figure-in-the-atlanta-schools-cheating-scandal-dies.html?_r=0. Retrieved 11 December 2014.
Fenton JJ, Jerant AF, Bertakis KD, & Franks P. The Cost of Satisfaction: A National Study of Patient Satisfaction, Health Care Utilization, Expenditures, and Mortality. Arch Intern Med. 2012; 172(5):405-411.
Gross R, Olson M, Gameroff M, Shea M, Feder, A, Fuentes M, Lantigua, M & Weissman M. Borderline Personality Disorder in Primary Care. Arch Intern Med. 2002; 162(1):53-60.
Whitmire R. The Bee Eater. New York, NY: Jossey-Bass. 2013.
©2014
All images public domain
medfly.org 