Data Analytics Model Gauges False-Negative Rate for COVID-19 Tests

A data analytics model could help researchers evaluate the false-negative rate of COVID-19 tests by comparing the sensitivity of different viral DNA test kits, according to a study published in Clinical Infectious Diseases.

Until enough Americans have received the vaccine, public officials believe that safely opening up schools, restaurants, and other public spaces will rely on widespread COVID-19 testing. As of June 2020, the FDA had granted emergency use authorization for more than 85 different viral DNA test kits, or assays, each with varying degrees of sensitivity and unknown rates of accuracy.

However, with no existing gold standard test for COVID-19, there is little data on which to judge these various tests’ usefulness to policymakers’ efforts to safely reopen businesses.

Researchers at Beth Israel Deaconess Medical Center (BIDMC) have developed a data analytics model to assess tests’ false-negative rate. The approach allows a comparison of the various assays’ clinical sensitivity.

“For getting back to business as usual, we all agree we’ve got to massively ramp up testing to figure out who’s negative and who’s infectious — but that’s only going to work optimally if you can catch all the positive cases,” said co-corresponding author James E. Kirby, MD, Director of the Clinical Microbiology Laboratories at BIDMC.

“We found that clinical sensitivities vary widely, which has clear implications for patient care, epidemiology and the social and economic management of the ongoing pandemic.”

COVID-19 tests are usually reported as simply positive or negative. However, the team noted that positive individuals can harbor drastically different amounts of viral load, depending on how long they’ve been infected or how severe their symptoms are. Viral load can vary as much as a hundred million-fold among individuals.

Researchers used data from more than 27,000 COVID-19 tests performed at Beth Israel hospital sites between March 26 to May 2, 2020. The team found that viral loads can be dependably reported.

“This helps distinguish potential superspreaders, at one extreme, from convalescent people, with almost no virus, and therefore low likelihood of spreading the infection,” said co-corresponding author Ramy Arnaout, MD, DPhil, Associate Director of the Clinical Microbiology Laboratories at BIDMC.

Researchers then estimated the clinical sensitivity and false negative rate for the in-house test, which was among the first to be implemented nationwide and considered among the best in class.

After analyzing repeat test results for the nearly 5,000 patients who tested positive allowed researchers to determine that the in-house test provided a false-negative in about ten percent of cases, giving the assay a clinical sensitivity of about 90 percent.

To estimate the accuracy of other assays, the team based their calculations on each tests’ limit of detection (LoD), defined as the smallest amount of viral DNA detectable that a test will catch 95 percent or more of the time.

The team showed that LoD can be used as a proxy to estimate a given assay’s clinical sensitivity. According to the researchers’ calculations, an assay with an LoD of 1,000 copies viral DNA per mL is expected to detect just 75 percent of patients with COVID-19, giving one out of every four people a false negative.

Researchers also showed that one test available today misses as many as one in three infected individuals, while another may miss up to 60 percent of positive cases.

“These results are especially important as we transition from testing mostly symptomatic individuals to more regular screening across the community,” said Arnaout.

“How many people will be missed—the false negative rate—depends on which test is used. With our model, we are better informed to ask how likely these people are to be infectious.”

Although not every COVID-positive patient missed by PCR and antigen tests will be infectious to others, some will – and this is reason enough to evaluate tests’ sensitivity, the team concluded.

“These misses will undermine public health efforts and put patients and their contacts at risk,” said Arnaout. “This must give us pause, and we really need to benchmark each new test even in our rush to increase testing capacity to understand how well they support our testing goals.”