How to Design a Comprehensive EHR Usability Assessment

What is EHR usability?

HIMSS defines EHR usability as “the effectiveness, efficiency and satisfaction with which specific users can achieve a specific set of tasks in a particular environment.”

The “usability” of a particular system can be assessed by identifying how well it meets criteria in nine key categories.

Simplicity

A simply designed EHR interface does not include information or visual elements that are unnecessary for completing administrative or clinical tasks. Simple interfaces reduce the likelihood physicians will be overwhelmed by the amount of data appearing on their screens.

EHR systems designed with simplicity in mind also highlight important information using visual cues and include navigation options that are easy to understand. Highlighting key information helps providers find relevant data more quickly. The design of the interface itself is clear, clean, and uncluttered.

Naturalness

According to HIMSS, natural EHR interfaces should include “screen metaphors that are familiar to everyday life, or commonly expected computer experiences for the clinician.”

Clinical workflows in a natural EHR interface are specific to the needs of clinicians. The interface is also intuitive and easy to learn to limit the burden of EHR training.

Consistency

In a consistent EHR interface, all parts of the application have the same look and feel. Terminology and data entry fields are also used and placed consistently, and providers are able to move seamlessly through different workflows. Consistent EHR interfaces promote continuity and reduce confusion for providers.

Forgiveness and feedback

Data entry errors happen. If providers make a mistake within the system, a forgiving application allows users to recover from errors easily. The system also provides appropriate and non-intrusive feedback to the user about actions they are about to take or have already taken to reduce the likelihood a user will take an unintended action.

The system also informs users when it is processing, indicates how long an action will take, and clarifies the purpose of the action.

Effective use of language

Language used within consumer-friendly EHR systems mirrors words clinicians use in practice, while also providing mapping to standardize codes and terms for data retrieval. List and entry-form choices are unambiguous. Overall, language within the EHR interface should read like natural English.

Efficient interactions

An EHR system that enables efficient interactions is designed to minimize the number of clicks or steps required to complete tasks.

An efficient EHR system also offers navigation options such as shortcuts to experienced users. Navigation methods minimize the need for user movements including scrolling and switching between typing and clicking the mouse to streamline EHR use for clinicians.

Effective information presentation

The design of the EHR interface uses clear fonts and visually-appealing elements to allow users to easily consume information.  

High-value data or abnormal values for clinical data are highlighted or presented in some way that conveys meaning. For example, a system may use red text or images to signal that a data element is abnormal and needs to be addressed urgently.  

Preservation of continuity

The EHR interface includes minimal screen changes and visual interruptions while providers are completing tasks, allowing users to focus on the content of the workflow rather than acclimating to a new environment after every action.

Minimized cognitive load

To minimize cognitive load, EHR data should be cohesively organized according to task without requiring users to access multiple screens simultaneously.  

EHR alerts should be concise, informative, and appropriate. Ensuring EHR alerts are only used in high-impact situations helps to reduce the likelihood of EHR alert fatigue, which can pose a threat to patient safety in some cases.

A consumer-friendly EHR system also performs calculations automatically for providers to reduce cognitive load and prevent human error.

Determining whether an EHR system possesses these qualities can be a challenge given the subjectivity of user experience. However, healthcare organizations can build off industry research when identifying potential avenues for conducting a comprehensive EHR usability assessment during and after implementation.

Key concepts to consider before launching EHR usability assessments

Health IT developers and healthcare organizations make decisions throughout the EHR lifecycle that affect a system’s degree of usability.

Federal regulators such as ONC can test an EHR system’s performance against key metrics prior to implementation, but the onus falls on healthcare organizations to determine whether a system meets the needs of its providers and unique environment.

In a 2018 report, Pew Charitable Trusts, AMA, and MedStar Health’s National Center for Human Factors in Healthcare identified a number of  ways healthcare organizations can ensure EHR usability testing accurately evaluates a wide range of functionality and potential scenarios.

First, the organizations suggested healthcare leadership consider all key tasks when planning an EHR usability assessment.

“Developer usability testing performed for certification focuses on EHR functions required by ONC,” wrote the organizations. “Some vendors develop test cases that include tasks to evaluate safety, but this practice is not pervasive. Test cases should also focus on more key tasks in which the use of these systems can affect safety.”

The organizations also advised that an EHR usability assessment be representative of the primary end-user, whether that is a nurse, a physician, or another qualified staff member.

“When possible, represent complete workflows that involve multiple types of clinicians participating in the scenario simultaneously,” the report advised.

As part of ensuring EHR usability assessments are representative of a healthcare organization’s end users, Pew recommends healthcare organizations consider the demands of its clinical workflows, environment, and any EHR-integrated third-party technologies. Healthcare organizations may also want to ensure usability assessments are clinically relevant.

Pew and its partners also stressed that assessing EHR usability at regular intervals is essential for accurately gauging EHR functionality.

“Various stages of the product life cycle, including how the product is modified by health care facilities and how software upgrades are implemented, can present different usability and safety challenges,” the organizations wrote.

Finally, Pew and its partners suggested healthcare organizations consider the socio-technical environment of their facility when assessing the efficiency and effectiveness of EHR features.

“Certification testing, conducted before implementation in healthcare facilities, focuses on the released EHR product and may not control for other factors that can influence safety,” the organizations cautioned. “For example, the type of training clinicians receive determines their knowledge of the EHR’s features, including how to order medications, diagnostic images, and lab tests efficiently and safely.”

“Additionally, the healthcare facility may make decisions during EHR implementation about how to organization information in the system, which affects how clinicians interact with the technology,” the authors continued.

With these concepts in mind, healthcare organizations can identify concrete goals and measures for a meaningful EHR usability assessment.

Pew and its partners recommend that healthcare organizations identify clearly definable concrete goals and measures that target a specific goal prior to beginning assessments. Additionally, the organizations suggested healthcare organization leadership ensure test cases are reproducible and unambiguous.

“Developing and using test cases that adhere to these criteria will provide greater rigor to the evaluation of clinician interaction with EHRs and can serve to better highlight specific usability and safety challenges in the design, customization, or use of products before patients are harmed,” maintained authors in the report.

EHR usability testing methods for healthcare organizations

HIMSS categorizes EHR usability testing in one of two ways: formative and summative.

Health IT developers are responsible for formative testing. Formative testing includes activities that help developers understand the user and clinical workflows, make iterative improvements to the product, and design the EHR interface in accordance with user needs.

Healthcare organizations perform summative testing. Summative EHR usability activities include expert review, performance testing, risk assessment, and one-on-one usability testing.

In combination, these activities can help healthcare providers evaluate efficiency, effectiveness, cognitive load, and other principles of EHR usability.

HIMSS recommends healthcare organizations test aspects of EHR usability in the following ways:

Efficiency

Expert review and one-on-one usability testing can be effective in measuring efficiency, according to HIMSS.

One-on-one usability testing can be performed in conjunction with an expert, intermediate, or novice end user. In one-on-one usability testing, end users are prompted to complete key tasks within the EHR interface and collect performance data during each task.

“Expert review is a human factors expert review of the product,” wrote HIMSS. “As part of the review, the usability specialist identifies areas in the product where the product conforms or fails to conform to Human Factors best practices.”

One-on-one usability testing requires close monitoring to ensure performance data is accurately recorded.

“Sessions are frequently recorded with special software that captures interactions with the graphical user interfaces and matches the interactions with time stamps,” stated HIMSS.

Users can record key indicators such as the time it takes to perform a particular task, the number of interactions necessary to complete the task, the number of screens visited to complete a scenario within clinical workflows, the number of times the ‘back’ button is used, and the time it takes to complete a set of instructions within the EHR system.

In 2017, Arcadia Healthcare Solutions measured the number of clicks a single provider needed to complete her clinical and administrative tasks throughout an average workday.

According to Arcadia Healthcare Solution’s research, the clinician needed 2,541 clicks to document 24 patient visits over the course of a 16-hour workday.

The results confirm provider concerns that EHR interfaces lack efficiency and require providers to perform a high number of interactions to fulfill reporting requirements and complete clinical documentation.

Recording the amount of time providers spend completing tasks within the EHR can help healthcare organizations identify particularly labor-intensive aspects of clinical documentation to guide workflow optimization.

Organizations that undertake similar examinations of routine EHR use may be able to identify areas for improvement or new workflow strategies that minimize cognitive fatigue.

Effectiveness

Risk analysis exercises are helpful for determining a system’s ability to avoid errors when completing clinical tasks.

HIMSS recommends using the Failure Modes and Effects Analysis (FMEA), or a topological risk analysis.

The FMEA risk management tool analyzes potential failure using three criteria: occurrence, severity, and detection.

To measure occurrence, experts must determine the cause of failures and the frequency with which they are likely to happen. Severity criteria measure the impact of failures on the end user or the patient, and detection criteria measure the likelihood that failures will be detected by the system.

Topological risk analyses allow healthcare organizations to identify risk elements including single-point failures and common-mode failures.  

“A single‐point failure would be any action by the clinician that results in harm, injury or death to the patient without a redundant safety check in place,” clarified HIMSS.

For example, if a provider mistakenly prescribed a drug to an allergic patient because drug allergy data was not displayed on the ordering screen, this would be considered a single-point failure.

Meanwhile, a common-mode failure is a safety incident in which multiple actions by the clinician stem from a single cause.

If a healthcare organization required prescribers to access multiple screens and memorize patient information to complete prescribing processes, errors occuring as a result of this ineffective process would be considered common-mode failures.  

“A usability rating process can be developed by adapting risk assessment methodologies to objectively evaluate the potential for user error,” states HIMSS. “Certain design factors can lead to user error which would have patient safety implications.”

One-on-one usability testing can also help to measure effectiveness. Experts or users can record the number or rate of errors, path taken to complete a task, severity of errors, and number of requests for help to measure effectiveness during usability testing.

Naturalness and consistency

To measure consistency, healthcare organizations can prompt users to conduct one-on-one usability testing using several key metrics. Healthcare organizations can measure the time it takes users to achieve expert performance, the number of icons users remember after completing a task, and the time spent undertaking manual processes.

Healthcare organizations can also compare difference in completion time for novice and experienced users to gauge how familiarity with a system contributes to successful use.  

Cognitive load

Measuring cognitive load is a complex process. Healthcare organizations may need to call upon cognitive psychologists to assist in administering assessments of cognitive load.

ONC’s Change Package for EHR Usability recommends healthcare organizations utilize the NASA Task Load Index to assess cognitive workload.

“The NASA Task Load Index (TLX) is a workload assessment tool that allows users to assess task load or burden based on the mental demand, physical demand, temporal demand, performance effort, and frustration of the human,” state ONC officials in the report.

“NASA TLX is widely used as a workload measurement tool across industries and is available as a printed PDF document or paper-and-pencil use, or as a mobile application.”.

AHRQ offers healthcare stakeholders a guide on how to use NASA TLX to measure cognitive load during EHR usability assessments.

User satisfaction

Finally, measuring user satisfaction is imperative for determining EHR usability. However, user satisfaction ratings are subjective and should be viewed accordingly.

Healthcare organizations can prompt users to complete tasks within the EHR and subsequently rate the ease and quality of their experience according to a point scale.

When researchers from the University of Missouri tested user satisfaction with different EHR note designs for a 2017 study, they asked participating clinicians to assign usability ratings to each design template based on the System Usability Scale (SUS).

SUS is a standard methodology that consists of 10 questions with five response options per question ranging from “strongly agree” to “strongly disagree.”

In addition to using SUS, researchers also prompted study participants to perform one-on-one usability testing with different EHR note designs.

“We simulated their utility for clinical practice by imposing time limits and by interrupting one of the tasks with a typical clinical interruption,” wrote researchers in the study report. “For each session, we recorded audio, computer-screen activity, eye tracking, and made field notes.”

Analyzing user performance data alongside SUS user satisfaction ratings helped researchers determine which EHR note design best met the needs of clinicians and promoted the highest level of clinical efficiency.

EHR usability can significantly affect clinical productivity, provider satisfaction, and patient safety. By observing and measuring provider interactions with EHR technology, healthcare organizations can get a sense of how well a technology, workflow, or application fits within their specific clinical environment.