Poor EHR Training, System Defects Hamper DOD EHR Implementation

The Department of Defense (DOD) has made progress in its MHS GENESIS EHR implementation, however health IT system defects and ineffective EHR training pose challenges to successful deployment, according to a new report from the Government Accountability Office (GAO).

The report is based on test reports, briefing materials, incident report tracking documents, interviews with relevant officials, and GAO-led discussion groups with 356 users at selected sites.  

MHS GENESIS operational testing at pilot sites revealed system defects. During initial and follow-on operational testing of MHS GENESIS, the Joint Interoperability Test Command identified 710 incidents, of which 409 had been closed. As of February 2021, 301 remained unresolved.

DOD took steps to improve system performance by closing an additional 13 identified test incidents, leaving 288 unresolved incidents as of May 2021.

DOD classified the 288 incidents according to five priority levels: critical, major, moderate, minor, and insignificant.

“According to the program’s test plan, critical incidents can result in mission failure, major incidents could cause partial failure, moderate incidents could result in the degradation of mission-related capabilities, minor incidents are noticeable but do not interfere with mission accomplishment, and insignificant incidents are informational,” the report explained.

Of all the unresolved incidents, 61 are considered a critical priority, 81 are considered a major priority, and 107 are considered a moderate priority.

However, despite some progress toward addressing MHS GENESIS incidents, DOD has not developed plans to conduct additional testing at future sites, the report noted.

“Without additional testing, the department lacks assurance that unresolved incidents will be fully addressed,” the report authors wrote. “As a result, known incidents that have not been addressed could lead to system deployment challenges at future wave sites.”

GAO called for the Secretary of Defense to direct the Program Executive Officer of Defense Health Management Systems to develop a method for retesting the incidents identified by testers to ensure that any system errors have been resolved.

Additionally, test results and discussions with system users revealed clinician training and communication challenges regarding the EHR implementation.

For example, clinicians said that EHR training modules were not consistent with the live system. Users also reported that there were too many system changes to keep up with and that they were not sufficiently informed regarding changes to the EHR.

“As a result, users were unaware of important changes to their roles or business processes, or to system revisions and improvements,” the report authors explained. “These challenges could hinder users' ability to effectively use the system, impede their knowledge of new workflows, and limit the utility of system improvements.”

GAO called on the Secretary of Defense to direct the Program Executive Officer of Defense Health Management Systems to improve the effectiveness of MHS GENESIS training by ensuring it is relevant to user roles.

Additionally, GAO called for the Secretary of Defense to direct the Defense Health Agency Chief Health Informatics Officer to ensure users are aware of EHR system changes and to monitor users' awareness of changes.

Since the implementation began in 2017, DOD has deployed the EHR system to 41,600 of about 201,400 planned users.

DOD plans to fully deploy the system to the approximately 159,800 remaining users, by the end of December 2023.

The report noted that this will require DOD to deploy the system to users at the remaining sites at a pace that is more than five times faster than the pace of previous deployments.