ML Models Can Help Optimize COVID-19 Hospital Admission Decisions

A new study published this month in npj Digital Medicine shows the positive impact of the multisite implementation of a workflow-integrated machine-learning (ML) system to predict short-term risk and optimize COVID-19 hospital admission decisions.

During the coronavirus pandemic, hospitals and emergency departments were placed under significant strain, and existing challenges in these settings before COVID-19 were exacerbated.

For example, emergency departments are routinely tasked with distinguishing patients who require hospital resources from those who may be safely discharged to the community. However, these determinations became even more challenging because of the novelty and variability of COVID-19, the study authors stated.

Further, relevant clinical decision support (CDS) tools during this time were limited. Thus, the researchers sought to create a CDS system embedded in EHRs that could estimate short-term risk for clinical deterioration in patients with or under investigation for COVID-19.

They gathered data for model development from a cohort of 21,452 patients who visited one of five participating emergency room study sites from March 1, 2020 to July 20, 2021. Researchers prospectively validated the models using 15,670 of these visits, with 4,322 occurring before CDS implementation and 11,348 after. They then measured model performance and multiple patient-oriented outcomes.

Patients over 18 who were designated as persons under investigation (PUI) for COVID-19 were included in the study. PUI status was defined as “having active isolation orders in the EHR at the time of [emergency department] disposition.” Patients who were not under suspicion of having COVID-19, including those who underwent asymptomatic testing for the virus, were excluded from the study.

The CDS translates model-generated risk for critical care needs within 24 hours and inpatient care needs within 72 hours into interpretable COVID-19 Deterioration Risk Levels. These risk levels were made viewable within emergency department clinician workflow and used for clinical decision-making.

Risk predictions were generated using data routinely stored in the EHR during emergency department care, including patient demographics, chief complaint(s), active medical problems, vital signs, routine laboratory results, markers of inflammation, SARS-CoV-2 status, and respiratory support requirements.

Overall, the incidence of critical care needs within 24 hours and inpatient care needs within 72 hours were 10.7 percent and 22.5 percent, respectively, and were similar across study periods. Total mortality, one of the key patient-oriented outcomes, remained unchanged across study periods but reduced among high-risk patients after CDS implementation.

ML model performance was excellent under all conditions, with area under the receiver operating characteristic curve (AUC), a statistical measure of usefulness, ranging from 0.85 to 0.91 for prediction of critical care needs and 0.80 to 0.90 for inpatient care needs.

The researchers stated that these findings indicate that ML models can reliably estimate risk for short-term clinical deterioration and that model output was translated into actionable advice within the existing clinical workflow for emergency clinicians. The models also provide insights that can inform clinical practice and interpretation of similar models in the future, they added.