Broad use of EHR voice assistants still years away
EHR voice assistants aren't much more than a Siri-type interface to the patient's healthcare record right now. But vendors and clinicians see big things for the tech.
Before Yaa Kumah-Crystal, M.D., opens the door to the next patient exam room, she routinely pulls out her mobile device and opens the Epic Haiku mobile app.
Instead of scrolling through the patient's chart, Kumah-Crystal will often press a button and say: "Hey Epic! Show me my last note." The voice command prompts Kumah-Crystal's last note on the patient to surface, an action that saves her from having to sift through the chart for the same information.
"Your last note is something you typically want to see. The detail gives you a good, general idea of what's going on with the patient and what your last frame of reference was," she said. "But if a patient hasn't been seen by you in six months and they have 20 other visits between, you have to do a lot of scrolling to find that. To be able to say, 'What's my last note,' and have that magically appear is golden."
Major EHR vendors, including Epic and Cerner, have embedded voice-based virtual assistants into their EHR product lines to automate routine tasks and reduce physician burnout. However, EHR voice assistants are still immature, only capable of performing command-and-reply functions instead of acting as an intuitive ambient documentation technology that captures patient-doctor communication, cues actions such as prescription ordering and automatically drafts the physician note. UCHealth CIO Steve Hess referred to this vision of EHR voice assistants as the holy grail.
That capability is still a ways off. It faces development hurdles such as background noise and its ability to hinder the technology's accuracy. Plus, Epic, Cerner and other EHR vendors rely primarily on a single vendor for voice technology, Nuance Communications, which could slow progress down, according to Jeffrey Becker, principal analyst at CB Insights.
The Hey Epic! EHR voice assistant responds to simple voice commands like 'show me my last patient note.'
Launching EHR voice assistants
The Epic team launched its first version of Hey Epic! in 2018 through its mobile app, Haiku, believing clinicians would already have most of the technology needed to use the new feature in their smartphones.
Since version 1.0, Epic has added more than 100 commands to Hey Epic!, including the ability to send messages to other physicians and cue up orders for follow-up tests and referrals, according to Seth Howard, vice president of research and development at Epic. Along with Nuance, Epic partners with 3M M*Modal, a speech recognition technology provider, to power Hey Epic! capabilities.
Cerner began testing its voice assistant, aptly named Voice Assist, at select healthcare organizations including Indiana University Health and St. Joseph's Health in 2020, according to Jacob Geers, director of voice experiences at Cerner. Geers' team spent 18 months building Voice Assist and put out a call last October for additional organizations to pilot the virtual assistant. The product is expected to be widely available in late 2021.
To be able to say, 'What's my last note,' and have that magically appear is golden.
Yaa Kumah-Crystal, M.D.Assistant professor of biomedical informatics and pediatric endocrinology, Vanderbilt University Medical Center
Geers said the growing maturity of natural language processing and Cerner's partnership with Nuance was critical in getting the capability off the ground, as was the vendor's adoption of the cloud. In 2019, Cerner selected AWS as its cloud provider, which provided access to more tools and capabilities it needed to introduce a voice assistant.
Despite Amazon's development of its own voice technology, including healthcare products Transcribe Medical, a speech recognition service, and Comprehend Medical, a HIPAA-compliant natural language processing service, Cerner plans to maintain its partnership with Nuance, which provides a prebuilt voice assistant package with appropriate medical terminology across different specialties, according to Geers.
"[Nuance] had it available in kind of a package that allows us to do what we call intent analysis," Geers said. "When someone requests something, they're able to extract the pieces and the parts of it that let us know where to go and take that request, what to do with it and what to serve back up to the user in the EHR."
CB Insight's Becker said Cerner's decision to continue partnering with Nuance points to how advanced its voice services are in healthcare.
"The fact that Nuance won that business tells you they're just that much further ahead than maybe anybody else could realistically catch up with in an amount of time that makes sense," Becker said.
Geers said building a voice assistant for healthcare is no simple undertaking, given the complexity of voices and sound in some clinical settings and the need for incredible accuracy.
"The ROI had to be there in order for this to reduce the time they spend interacting with the EHR," Geers said.
Geers said his team conducted studies to determine how a voice assistant might make the most impact by studying where clinicians were struggling with things like click fatigue. The team built Voice Assist to follow commands on navigating tabs within the patient chart and setting reminders to searching the chart and locating necessary documents.
While Cerner continues to partner with healthcare systems on Voice Assist, Epic is looking to expand where its voice assistant can be found.
In 2021, the EHR vendor wants to make Hey Epic! available on exam room desktop computers. The core technology is already built, and Epic will be working with healthcare systems to implement and test the product, according to Howard.
A starting point
Kumah-Crystal, an assistant professor of biomedical informatics and pediatric endocrinology at Vanderbilt University Medical Center, is a regular user of Hey Epic!, noting that it's on par with consumer voice technologies. But the tool is limited to basic commands, and Kumah-Crystal said she would eventually like to use the voice assistant for more specific commands such as "show me the last time this patient saw radiology for a fracture of the knee."
Indeed, although voice technology has progressed in recent years, it is still limited, even in the consumer space where accuracy isn't as critical.
"You see a lot of hype around voice assistants in general, not just in healthcare," Becker said. "You think of them as being very capable until you personally have to use one to do something and then you realize they are still very unintelligent and immature. So, usability is a long slog for virtual assistants."
Indeed, UCHealth CIO Hess, a Nuance and Epic customer, said he sees virtual assistants in their current state as being more work -- not less -- for clinicians.
More than 2,000 UCHealth doctors use Nuance's speech-to-text tool Dragon Medical One to dictate their notes. He said there is strong interest in voice assistants for transcription, but not for something like querying the patient chart -- at least not yet.
"If you're a facile user of the EHR, this is actually a step backward right now, to use voice, because it's just not made for the templates of the EHR," he said.
Hess believes EHR voice assistants will be extremely useful when they can be used as ambient listening tools that capture the provider-patient experience and take intuitive action such as automatically transcribing the doctor's note in the EHR and cuing up next steps.
"I think it's a two- to five-year journey, and hopefully by the end of five years, we'll actually have something that we're like, 'Hey, this works,' and makes the jobs of the doctors and nurses better," Hess said.
EHR voice assistants of the future
Epic's Howard echoed this future vision for EHR voice assistants.
Ideally, an ambient listening and documentation tool would mean that doctors and nurses could spend less time on the computer and more time with patients.
Howard said for the industry to get there, EHR voice assistants would need to mature greatly so that they could understand both clinicians and their use of medical terminology, as well as patients and the imprecise way they use language to describe clinical terms.
Howard put the problem like this: The voice assistant would need to fill in the gaps in ambiguous conversations, likely by mimicking human deductive reasoning to draw conclusions based on the conversation.
"It's a tough problem to solve and some folks would say it sounds a bit like science fiction at this point," Howard said. "But we're seeing some good early progress on prototypes, especially for targeted specialty visits like orthopedics follow-up care."
Indeed, Becker said a major goal for EHR voice assistants is to eventually document the entire patient visit to reduce physician burden, which he believes is a three- to five-year race to the finish for the vendors.
"I think that is where everybody is shooting for and will eventually get to -- where these virtual assistants are pulling results, entering orders, listening to the conversation and auto-creating the framework of the note," Becker said. "So that when the doctor goes in, the note is there, the orders are there. Then they will sign off on the note and sign off on the whole chart and you're good."
