Using Machine Learning to Enhance COVID-19 Prediction Models

Using an advanced machine learning technique, Brown University researchers are exploring how to improve COVID-19 predictive models.

“There’s an old saying in the modeling field that ‘all models are wrong, but some are useful,’” professor of applied mathematics and engineering at Brown and senior author, George Karniadakis, said in a press release.

“What we show here is that the major COVID-19 models were wrong and also not very useful — at least in terms of predicting the course of the pandemic. There was a lot of Monday-morning quarterbacking, but not a lot of accurate predictions.”

The research team examined nine prominent COVID-19 prediction models, all of which were a variation of the susceptible-infectious-removed” or SIR model. These models separated a sample population into two groups: those without the virus (susceptible), those with the virus (infectious), and those who had the virus but could no longer spread it (removed).

According to researchers, the major downfall of COVID-19 models was that they treated key parameter values as being fixed over time, despite the fact the data could shift dramatically in the real world.

“The community transmission rate of the virus varied widely depending upon mask use, business closings and re-openings, and other measures. Hospitalization rates changed over time as the availability of hospital beds shifted. And the death rate changed with new treatments,” the press release stated.

“All of these evolving factors changed the trajectory of case rates and deaths, but prominent models held these parameters steady in time, which led to poor predictions.”

To capture the changing parameters, researchers used physics-informed neural networks (PINNs) — a machine learning technique developed by Karniadakis and his colleagues. PINNs are neural networks like those used to recognize images or transcribe speech to text. Equipping the system with equations, Karniadakis and the team used PINNS to calculate how pathogens spread.

“Considering the fact that pandemics evolve in time and there is continuous collection of data, PINNs can be retrained as new data is collected and update the models over time with inferred parameters,” said Ehsan Kharazmi, a visiting scholar at Brown and study’s co-lead author.

“The computational time needed for retraining PINNs with new data is relatively short compared to the time-scale of pandemic evolution.”

Researchers fed the machine learning model real-world data from New York City, the states of Rhode Island and Michigan, and national data from Italy, allowing PINNs to predict future outcomes.

In January 2021, the team made predictions for the next six months based on the time-adjusted parameters. Comparing the model’s predictions to actual cases rates, researchers found that the case rates from January through June 2021 fell within the uncertainty window predicted by the models.

According to researchers, their findings suggest that while no models can accurately capture all the data that impacts a pandemic, models that can adjust key parameters on the fly could make more useful predictions.

“The inferred models using PINNs can be used to assess possible future trajectories by tweaking the model parameters,” Kharazmi said. “This can provide some insights for making or adjusting policies.”