Site-Specific ‘Discount’ EHR Usability Testing Can Improve Clinician Experience

Site-specific EHR usability testing with a small group of clinicians can help optimize the clinician experience without investment in usability experts, according to a study published in Applied Clinical Informatics.

While EHR certification requires vendors to perform user testing, site-specific implementations do not require testing. Healthcare organizations customize EHR implementations, potentially introducing usability issues.

Researchers developed a site-specific EHR usability program and evaluated thirteen new features available in the EHR throughout three user test events. Each event contained up to six end-users who measured the features for user efficiency, effectiveness, and satisfaction.

The team identified 33 usability concerns (2.5 problems for each feature tested). The researchers determined that more than 70 percent of the usability concerns were correctable.

For instance, the original clinical pathway to guide users toward the most appropriate vascular access method did not account for alternative workflows revealed in the user test event.

“Complete redesign of this feature likely prevented a large number of unfortunate user experiences,” the study authors noted.

For the first test event, the researchers used an application specifically designed for user testing called Morae. However, they found that the software was complex, took time to learn and configure, and was difficult to use with remote testing. The team changed to an online meeting platform for the second and third test events, which improved qualitative findings.

“The move to a lighter, simpler platform meant less quantitative data but also simpler configurations, less prep time for each testing session, and more rapid test cycling,” the authors wrote. “The forced change from Morae to Zoom served to improve our program by shifting the focus from quantitative to qualitative findings, a key component of discount usability.”

The researchers noted that many healthcare organizations might not have staff with training or experience in usability evaluations. However, ‘discount’ usability testing methods like the one outlined in the study could help improve the clinician experience without dedicated usability specialists.

“We acknowledge that our user testing program is imperfect, but our approach, using elements of discount usability, has served to improve the user experience and may provide a template for other health systems to implement site-specific user testing programs of their own,” the study authors said.

“Further research may clarify the best practices and additional ways to make site-specific user testing scalable so that users benefit from testing of all aspects of an implementation,” they added. “We did not usability test every implemented feature. Work which identifies which types of features introduce the most serious usability concerns would help sites focus testing efforts to maximize value.”

Additionally, testing outside formal “test events” may help organizations scale up a testing program by simplifying recruitment.

“Future work should define the value of testing in other settings, such as simulation labs and EHR help centers,” the researchers pointed out.

This study has important limitations.

First, the researchers made two important changes after the first test event. They noted that adding a financial incentive likely altered recruitment patterns and user testing incentives, and switching to an alternative recording software could have changed quantitative usability test results.

Second, the study was limited to user testing with providers.

“Nurses may have different usability concerns, and the yield from a discount usability approach may differ from that described in this report,” they emphasized.

Lastly, the study occurred at a single center, which limits its generalizability.