Generative AI tool shows promise in nephrology triage

New research shows that generative AI has the potential to support triage processes for nephrology subspecialty services; however, there is room for improvement.

AI is proving valuable in enhancing healthcare service access and utilization. Various studies underscore the fact that AI tools show promise in diagnosing complex conditions, predicting patient outcomes and prognoses, and streamlining discharge processes.

There is an urgent need to enhance nephrology care in the U.S., which concerns the study and treatment of kidney disease. According to the National Kidney Foundation, kidney disease is widely prevalent, affecting an estimated 35.5 million U.S. adults. Kidney disease patients face a higher risk of early death, and thus, timely access to care is crucial.

For the study published in Scientific Reports, Mayo Clinic researchers examined whether ChatGPT, a large language model, could be used to triage nephrology cases through simulated real-world scenarios. Two nephrologists created 100 patient cases designed to reflect a wide array of clinical conditions commonly encountered in nephrology. The cases included acute kidney injury, chronic kidney disease, diabetic kidney disease, kidney stones and glomerular diseases.

The researchers fed ChatGPT the data associated with each case, instructing it to provide clear and concise responses to whether a nephrology consultation was required for each case and recommendations for the most suitable subspecialty clinic.

In the first round, ChatGPT correctly determined the need for nephrology consultation in all simulated cases and correctly triaged patients to the most suitable nephrology subspecialty in 99 cases. In the second round, ChatGPT correctly identified the need for a nephrology consult in 99 simulated cases and correctly triaged patients to the most suitable nephrology subspecialty in 96 cases.

"Our study found that ChatGPT achieved an overall accuracy of 99.5% in determining the need for nephrology consultation across 100 simulated cases," the researchers wrote. "The AI performed with a high degree of clinical accuracy, appropriately identifying the necessity for nephrology referrals."

However, in a few cases, the tool directed patients to less appropriate subspecialties than needed. For instance, it recommended an electrolyte disorders clinic for a cancer patient experiencing renal electrolyte complications, the researchers noted.

"These errors highlight the potential gaps in AI's understanding of the nuances in clinical scenarios where patient conditions may span multiple specialty areas," the researchers wrote.

Further, they noted various concerns related to AI use in healthcare, including data privacy challenges and the risk of bias. With troves of personal health information transmitted through AI tools, HIPAA violations and data breaches could become more common. Not only that, but biases stemming from AI tools could widen existing care disparities.

Though the researchers highlighted the need for secure data-handling practices and robust monitoring mechanisms, it is unclear how closely healthcare AI stakeholders will follow responsible AI development guidelines in the future. Last week, President Donald Trump rescinded several executive orders passed during the Biden administration, including one regarding safe and trustworthy AI development.

The 2023 order provided guiding principles and priorities for AI development, including mitigating bias in AI systems and protecting Americans' privacy.

Anuja Vaidya has covered the healthcare industry since 2012. She currently covers the virtual healthcare landscape, including telehealth, remote patient monitoring and digital therapeutics.