Are Consumers Left Behind in Healthcare’s Generative AI Talks?

In the throes of the insurgence of generative AI, healthcare has a huge blind spot: consumers.

In a new report from Deloitte, experts found that most healthcare organizations are leaving behind the patient when considering how artificial intelligence can fit into their technology strategies moving forward. While most hospitals and health systems are focusing on data quality and availability, legal and regulatory compliance, and security and privacy concerns, less is being made of how generative AI will affect the healthcare consumer.

“Consumers have shown that they’re willing to engage with generative AI on their health care journeys,” a group of six Deloitte experts wrote in a blog post about the report. “As health care organizations begin to integrate generative AI into their workflows, taking a holistic, institutional approach may help achieve a successful implementation at an enterprise level.”

Generative AI surged onto the healthcare scene in late 2022 when tools like ChatGPT and similar large language models pushed healthcare executives and tech developers alike to assess how the tools could streamline the industry. But as stakeholders look for applications for the new AI models, some blind spots may be emerging.

The Deloitte 2024 Health Care Generative AI Outlook survey asked 60 healthcare executives about their strategies and considerations when looking at and implementing generative AI. Well over 70 percent of executives are laser-focused on the data aspect of implementing gen AI—data quality and availability, regulatory compliance, and security concerns—but the consumer is being left behind in those conversations.

Only 50 percent of executives said they are focused on building patient trust to share data, a crucial component to making generative AI models work, while the same proportion said they are focused on equitable access to gen AI-driven solutions. Even fewer (45 percent) are focused on patient education about AI and associated risks, Deloitte found.

To be sure, separate data has shown that consumers are enthusiastic about generative AI in healthcare. A November 2023 report from Deloitte showed that 53 percent of all healthcare consumers think generative AI can improve access to healthcare, and 46 percent said it could potentially improve healthcare affordability. Among those who’ve used generative AI before, those numbers are even higher—69 and 63 percent, respectively.

But that enthusiasm might not last long, at least not if organizations continue to leave consumers out of the generative AI conversation, Deloitte researchers contended.

“With less of a focus on what’s important to the consumers, health care organizations may find that trust and engagement levels drop,” the six researchers wrote in this most recent report.

Indeed, patient trust in generative AI is already 50/50 at best.

In October 2023, data from Propeller Insights gathered on behalf of Carta Healthcare found that 49 percent of patients were comfortable with their healthcare provider using AI in the healthcare setting, while 51 percent were not. Another study from May 2023 found that trust in AI chatbots, largely powered by the generative AI and large language models explored in the Deloitte report, is around 50 percent.

The November Deloitte report showed that healthcare consumers need transparency when AI is being used. Four in five healthcare consumers told Deloitte that it’s important or extremely important that their healthcare provider let them know when they are using generative AI for their healthcare needs.

That means healthcare organizations need to know their consumer before generative AI can realize its fullest potential in the medical setting, the Deloitte researchers said in this latest report.

“Organizations should actively engage consumers to understand their most critical pain points and understand what AI solutions they are willing to use,” they advised. “This can be achieved by gathering direct consumer input and conducting focus groups while iteratively building, testing, and deploying new products. The process should be led with transparency, equity, and collaboration.”

Healthcare consumers aren’t organizations’ only blind spots when it comes to generative AI, the report noted. Greater attention to eliminating bias in generative AI, plus considerations for workforce management, is also needed as organizations explore more use of the technology.