What Is Non-Response Bias in Patient Satisfaction, CAHPS Surveys?

How much does participation bias impact patient satisfaction surveys?

Participation bias is a key consideration for all statisticians. It is an important limitation they must all examine before drawing conclusions. But to what extent does non-response bias impact surveying in healthcare?

Data suggests several key patient populations are excluded from patient satisfaction surveys, most of which are vulnerable or otherwise underserved.

A 2011 study published in PLOS One examined the specific groups that tend not to respond to patient satisfaction surveys. Patients with substance use disorder (SUD), language barriers, cognitive impairments, and partial or total blindness are less likely to respond to patient satisfaction and CAHPS surveys than those with fewer needs.

A separate study published in BMC Health Services Research found similar results. Older patients, males, and those with public insurance faced more barriers in responding to patient satisfaction surveys than their younger, female, and privately insured counterparts.

Patients who fit these demographic criteria tend to face more social determinants of health, or social factors that hamper their interactions with the healthcare industry. These patients may not understand the questions asked in a patient satisfaction survey or lack the self-efficacy and activation to answer a survey.

To be clear, demographics are not the sole factor that influences whether a patient does or does not respond to a satisfaction survey. Some older, publicly insured males may answer a patient satisfaction survey, even though the data suggests they are unlikely to.

Personal characteristics also play a role here, although they are more difficult to measure, the BMC Health Services Research authors said.

“The propensity to respond to surveys of patient satisfaction is likely multifactorial, and underlying patient characteristics or personality traits (other than general measures such as age or sex, for example) that lead to response (or nonresponse) remain largely speculative and difficult to assess,” they wrote.

However, understanding the demographic data, in addition to having a strong understanding of patient motivation and characteristics that indicate a survey responder, will help organizational efforts to garner high response rates.

How can organizations prevent non-response bias?

Ideally, non-response biases would not be an issue for healthcare organizations and patient satisfaction surveying. However, the data suggests most organizations do see at least a modest bias in their response rates, which in turn influences their practice decisions.

The first step to reducing the impacts of non-response bias is preventing it to begin with.

Offering surveys in the most common languages spoken at the organization; conducting more outreach to all patients, especially those less likely to respond to the survey; writing surveys in basic language to account for limited health literacy; and making surveys available in the clinic, via telephone, over the smartphone, and online may help close response gaps.

Organizations may decide which initiatives are best suited for their needs based on their non-respondent populations. However, organizations must be wary of where there is no immediate solution to non-response, the PLOS One article authors suggest.

“For cognitive limitations, mental illness or substance abuse, proxy respondents may be considered,” they wrote. “Proxy respondents usually provide reliable factual information and cause limited biases, but whether this holds also for subjective assessments such as satisfaction with health care requires further study. Health surveys do require a level of cognitive ability and motivation that may be out of reach for a substantial proportion of the target population. However strategies to improve participation can be implemented only if potential barriers are identified.”

Because of those limitations, it may be difficult for an organization to entirely rid their patient satisfaction surveys of non-response bias. Therefore, it will be important for organizations to be transparent about their satisfaction survey methodologies.

“Broadly, when any survey is widely used as the basis for an external quality indicator or comparison tool, the importance of utilizing sound and transparent methodologies in the development, the administration, and the interpretation of the survey becomes paramount,” the BMC Health Services Research article stated.

If a patient satisfaction survey will be informing a certain practice decision, it will be essential for survey administrators and other practice leaders to be up front with patients, other staff, and themselves about possible biases and the limitations they impose.