Epic Sepsis Prediction Model Lacks Clinical Decision Support Timeliness

While the accuracy of a proprietary sepsis prediction model from Epic outperformed other tools at higher threshold predictions, poor timeliness limits its application for clinical decision support (CDS), according to a study published in JAMA Network Open.

Researchers tested the Epic Sepsis Prediction Model (SPM) against three existing sepsis prediction tools: Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), and quick Sepsis-Related Organ Failure Asessement (qSOFA).

The retrospective cohort study included all adults admitted to five acute care hospitals in a single health system between June 5, 2019, and December 31, 2020.

Although the SPM demonstrated better balanced accuracy and specificity for sepsis at higher-threshold predicting sepsis score (PSS) (8 to 10), it also missed a higher share of true cases and was less timely than SIRS and SOFA.

Initial clinician action indicating suspicion of infection occurred a median time of 68 to 145 minutes before threshold score when using higher, more accurate PSS thresholds.

“Poor timeliness combined with increased score complexity and lack of transparency of the SPM epitomizes its major flaw: it appears to predict sepsis long after the clinician has recognized possible sepsis and acted on that suspicion,” the study authors wrote.

This finding is consistent with prior research, indicating a lack of plausible clinical benefit of the Epic sepsis prediction model.

At higher PSS scores (8 to 10), the SPM could have only identified 12.9 to 19.7 percent of patients in a clinically relevant time before clinician action.

There was also a high proportion of patients at higher PSS thresholds with confirmed sepsis who never reached a threshold score (14.8 percent to 21.5 percent for scores of 8 to 10).

“Although setting higher PSS thresholds decreased false-positive rates, it also resulted in higher than acceptable false-negative rates and amplified problems with timeliness of detection,” the authors noted. “These findings suggest that the SPM has limited potential to shorten time to clinician action compared with alternative criteria.”

Given the existing literature regarding the importance of early antimicrobial administration, patients who would benefit most from early recognition are those at the highest risk for poor outcomes.

“Development of a prediction tool that accurately captures this high-risk group in a timely manner should be the focus of future model development,” the researchers emphasized.

Epic has updated its sepsis algorithm to version 2.0 in response to critical evaluation of the model in a clinical setting. However, it is yet to be verified whether updating the sepsis definition in the new tool will address timeliness.