ChatGPT demonstrates promise for digital pathology

Researchers from Weill Cornell Medicine and Dana-Farber Cancer Institute have developed ChatGPT-based tools to improve information retrieval and bolster software use in digital pathology.

The research team emphasized that generative AI tools have the potential to transform medical research, but that the integration of these tools -- such as large language models (LLMs) -- presents unique integration challenges in rapidly evolving specialties like digital pathology, which requires clinicians to derive diagnostic and treatment insights from images of tissue samples.

The researchers further noted that the use of ChatGPT in healthcare can be useful for certain information retrieval tasks, but struggle in contexts where more accurate, specific responses are required.

"LLMs are good for general tasks, but they aren't the best tools for getting useful information for specialized fields," explained lead study author Mohamed Omar, MD, assistant professor of research in pathology and laboratory medicine and a member of the Division of Computational and Systems Pathology at Weill Cornell Medicine, in a news release.

To overcome this challenge, the research team used a custom version of ChatGPT deployed at Dana-Farber known as GPT4DFCI.

"General LLMs have two major problems. First, they often provide lengthy generic responses that don't contain useful information," Omar stated. "Second, these models can hallucinate and make things up out of nowhere, including literature citations. This is especially bad in specialized fields like digital pathology and cancer biology, for example."

The augmented version of GPT4DFCI is designed to address these issues by pulling from a comprehensive, domain-specific database of digital pathology research from 2022 onward. Using a technique called retrieval-augmented generation, the tool can access information from 650 publications and 10,000 pages of literature in response to pathologists' queries.

"We could ask this new system to catch us up on many specific topics or techniques in digital pathology and get results in seconds, with a level of detail, depth and summarization that doesn't exist in current scientific literature tools or search engines," noted corresponding author Renato Umeton, Ph.D., director of artificial intelligence operations and data science services, informatics and analytics department at Dana-Farber.

When tested, GTP4DFCI's responses were more relevant and precise than those of a general GPT-4. In addition to its accuracy, the custom model also provided links to the specific publications used to generate its responses and did not hallucinate.

The researchers underscored the tool's potential for clinical decision support in pathology.

"My hope is that this will be a catalyst for more domain-specific tools in other fields of medicine or medical research," Omar said.

The research team also created a tool that integrates ChatGPT with PathML, a software library for histopathology analyses. PathML is useful for exploring complex imaging data, but requires familiarity with the programming language Python, which many pathologists lack.

"Pathologists or scientists without prior coding experience often find PathML very challenging to use for image analysis tasks," Omar explained.

The researchers sought to alleviate this burden using the combined tool, which allows users to type in their histopathology analysis queries and receive step-by-step coding instructions for PathML.

"Our research shows that, when combined with the proper information retrieval techniques, ChatGPT and safeguarded AI tools, like GPT4DFCI, can be extremely effective in supporting basic researchers," Umeton said. "These tools are helpful even across very complex topics that need extremely precise answers, like digital pathology."

Shania Kennedy has been covering news related to health IT and analytics since 2022.