Viewpoint: Pandemic Promise of Machine Learning Falls Short  

With the recent advances in machine learning algorithms, healthcare professionals had high expectations for the technology during the COVID-19 pandemic.

However, according to an analysis published by a Northwestern University Feinberg School of Medicine research team, these expectations have been mostly unrealized except for a few notable successes. In their viewpoint analysis, the researchers reflected on the underlying reasons to shift the approach from reactive to proactive machine learning, which involves the use of algorithms and statistics to identify patterns in data, to better utilize the technology.

Throughout the pandemic, hundreds of image-based machine learning models have been developed. However, very few models have achieved widespread clinical use for COVID-19 detection and forecasting due to data bias, data shift, and ethological limitations.

According to the research team, one significant success of machine learning during the pandemic involved the use of reinforcement learning to create more effective COVID-19 testing for travelers entering Greece.

“Observing that population-level epidemiologic metrics poorly identified infected asymptomatic travelers, investigators collected traveler-specific features, including age, sex, and travel history, to stratify 6,084,954 travelers into risk phenotypes with higher resolution than merely country of origin,” the viewpoint analysis stated.

“Their reinforcement-learning system (known as Eva) allocated scarce polymerase chain reaction (PCR) tests to maximize detection of infected asymptomatic travelers among those tested (exploitation), and automatically provided feedback that enabled Greece to target data collection to improve risk-estimation precision for undersampled groups (exploration).”

In this example, the machine learning method was able to provide data recommending COVID-19 test distribution to maximize the number of infections detected, even in asymptomatic cases.

The reinforcement-learning system significantly improved testing efficiency during peak travel compared with random testing. Additionally, the system informed Greece’s policy for “gray listing” high-risk countries to require travelers from those countries to provide proof of a negative PCR test before arriving in Greece.

While healthcare has begun adopting level 1 proactive machine learning, the research team explained that moving towards level 2 could open new avenues for the technology.

“For example, conventional clinical trials are expensive and often require large sample sizes, making their initiation and completion especially challenging for rapidly changing health care conditions such as the pandemic. An emerging form of adaptive platform trials has drawn increasing attention during the pandemic,” the research team said.

“The Randomized, Embedded, Multifactorial Adaptive Platform (REMAP) originally established for community-acquired pneumonia was pivoted to conduct trials for multiple COVID-19 treatments simultaneously and showed promising results for tocilizumab and sarilumab in critically ill patients.”

In REMAP, response-adaptive randomization can identify patients who could benefit from emerging treatments and interventions.

Additionally, level 2 proactive machine learning also allows augmented data preparation, especially for data that may be hard to extract.

A lot of key information regarding COVID-19 sequelae comes from patient-authored tests, which are not analysis-ready. With natural language processing, the technology can crowdsource patient perspectives into the level 2 feedback loop to augment data preparation and refine the downstream analysis, enabling data insights regarding treatment options.

According to researchers, the limited contributions of machine learning to address challenges created by COVID-19 promoted a reexamination of best practices during the pandemic and in the future.

“The pandemic, and the attendant need for adapting to the rapidly evolving landscape of health care, has acted as a stress test for ML. Understanding successes and unrealized opportunities not only highlights the nonrepresentative issues of the underlying data but also reveals the need to move from reactive toward more proactive ML,” the analysis stated.

“This fresh approach could lessen the potential influence on ML of data issues under challenging and evolving situations and may allow ML to coevolve with data and meaningfully influence more health care decisions and policies.”