Johns Hopkins Machine-Learning Tools Predict Risk of ICU Delirium

Johns Hopkins University researchers have developed machine-learning (ML) algorithms that can detect the early warning signs of delirium and predict which patients will be at high risk of delirium at any point during an intensive care unit (ICU) stay.

The press release states that over one-third of patients admitted to the hospital and 80 percent of ICU patients develop delirium at some point during their hospital stay. Delirium can cause inattention, confusion, paranoia, agitation, and hallucinations in patients, which can worsen patient outcomes.

ICU delirium occurs more frequently in older and sicker patients and can lead to prolonged hospitalization, future dementia, and death, according to the press release. Anti-delirium interventions, such as care bundles, earlier-than-usual physical and occupational therapy, and medication changes, can be effective but are not used on every patient because of limited time and resources and the sometimes unpredictable needs of these patients.

To address this, the researchers set out to build delirium prediction models.

“Being able to differentiate between patients at low and high risk of delirium is incredibly important in the ICU because it enables us to devote more resources toward interventions in the high-risk population,” said Robert Stevens, MD, associate professor of anesthesiology and critical care medicine at the Johns Hopkins University School of Medicine and senior author of the study, in the press release.

The researchers applied artificial intelligence (AI) algorithms to a publicly available dataset containing data on 200,000 ICU stays at 208 hospitals across the US. With this data, they developed two ML models to predict delirium risk.

The first, a static model, uses a single snapshot of patient data taken shortly after admission and uses information about age, the severity of illness, other diagnoses, physiologic variables, and current medications to make predictions.

The second, a dynamic model, monitors patient data over the course of hours and days following admission and throughout a patient’s stay, using information like repeat blood pressure, pulse, and temperature readings to continuously generate the patient’s risk of delirium over the next 12 hours.

“The underlying idea was that this routinely collected data stored in patients’ electronic health records contains signatures that are associated with delirium risk,” said Kirby Gong, a recent master’s degree graduate from the Johns Hopkins Department of Biomedical Engineering and first author of the study, in the press release.

Following model development, the researchers tested each model on two other data sets from a Boston hospital that contained information on over 100,000 ICU stays. They found that the static model achieved an area under the receiver operating characteristic curve of 0.785, indicating that it could accurately predict which patients would experience delirium 78.5 percent of the time. The dynamic model achieved an even higher performance score and was able to predict delirium-prone patients up to 90 percent of the time.

The researchers are now testing the models on historical data from Johns Hopkins Medicine ICUs. Moving forward, the research team plans to design a clinical trial to test the models in patients newly admitted to the ICU rather than relying on retrospective data. They are also applying similar AI and ML approaches to predict other events in critical care medicine, such as heart failure, stroke, and pulmonary embolism.

“For a lot of these physiological transitions, we think that there are early warning signs that may not be obvious to a clinician but can be picked up on using the kinds of artificial intelligence-supported pattern analysis that we used here,” explained Stevens.

These models join several other efforts to leverage AI to predict clinical risk.

In June, researchers showed that clinical risk prediction models for sepsis, delirium, and acute kidney injury (AKI) achieved high performance when implemented into live clinical workflows at multiple hospitals.

Typically, risk prediction models use retrospective data. But because of the lack of interoperability in legacy systems used to store retrospective data and differences in the prevalence of various clinical risk events, models using data from live workflows can be more accurate than those using retrospective data.

Overall, the performance of the prediction models using data from live clinical workflows was similar to that of the models using retrospective data. However, a cross-hospital evaluation showed severely reduced prediction performance, indicating significant limitations in developing generic prediction models for use in different hospitals. High performance was achieved as a result of the calibration step that utilized specific hospital data, the researchers noted.

However, they said more research focused on evaluating the detailed clinical outcomes of prediction models in medical practice is needed.