The fastest test being used to diagnose people infected with the coronavirus appears to be the least accurate test now in common use, according to new research obtained by NPR.
Researchers at the Cleveland Clinic tested 239 specimens known to contain the coronavirus using five of the most commonly used coronavirus tests, including the Abbott ID NOW. The ID NOW has generated widespread excitement because it can produce results in less than 15 minutes.
But the ID NOW only detected the virus in 85.2% of the samples, meaning it had a false-negative rate of 14.8 percent, according to Dr. Gary Procop, who heads COVID-19 testing at the Cleveland Clinic and led the study.
"So that means if you had 100 patients that were positive, 15% of those patients would be falsely called negative. They'd be told that they're negative for COVID when they're really positive," Procop told NPR in an interview. "That's not too good."
Procop says a test should be at least 95% reliable.
Procop, who plans to publish the study soon, chairs the Commission of Science, Technology and Policy for the American Society for Clinical Pathology.
Although the paper has not yet gone through the traditional peer-review process, Procop says the findings have been carefully reviewed at his hospital and he is confident in the results.
In a statement, Abbott defended the test's reliability.
"ID NOW performs as expected and we have confidence in the performance of the test," the statement said.
Abbott said any problems with the test could stem from samples being stored in a special solution known as viral transport media before being tested, instead of being inserted directly into the ID NOW testing machine. As a result, the company recently instructed all users to only test samples put directly into the machines.
"When a direct swab is used, the test is performing as expected," according to the company statement.
Abbott did not provide a scientist to discuss Procop's study, but instead referred NPR to a doctor in private practice using the machines.
"We feel that the test is very accurate because we're using it the way it's supposed to be used" by placing samples directly into the machine, said Dr. Warren Wollin, senior medical director at Physicians Immediate Care, which operates urgent care clinics in Illinois.
"So many patients come in and have so much anxiety because they have minor symptoms. And to be able to tell them right away gives them such relief," Wollin said.
The Cleveland Clinic's Procop acknowledged that all of the samples in his study were stored in viral transport media before being tested. But Procop argues the company should back up its claim with data that this could affect the test's accuracy, especially if the test is being used in settings where people may not be aware of possible false negatives.
"They need to prove to us that they can actually deliver a sensitivity greater than 95% [if samples are placed directly in the machine], or else we'll be putting citizens at risk," Procop says.
Based on his study, Procop said his hospital has stopped using the test to screen patients being admitted for care. The hospital also stopped using another test, called the DiaSorin Simplexa, because it only detected 89.3% of infections in his study, Procop said.
"If you have patients coming into hospital and you're going to put them into what has been determined to be a COVID-free ward, you have to have the most sensitive test available," Procop says. "Because once you put somebody with COVID into a COVID-free ward, it's no longer a COVID-free [ward] any more. It's your new COVID ward."
The Cleveland Clinic plans to rely on other tests that performed better in the study, Procop said. That includes the test developed by the Centers for Disease Control and Prevention, which detected 100% of positive samples. Another test, made by Roche, detected 96.5% of positive samples. The fifth test in the study, made by Cepheid, detected 98.2% of infected samples, Procop said. The Cepheid test produces results in less than an hour.
Abbott is shipping 50,000 coronavirus tests every day for use on 18,000 ID NOW testing machines in doctors' offices, clinics and hospitals around the country. The company hopes to boost that to at least 2 million every month by June.
Still, Procop, like Wollin, believes the Abbott test can be useful in many settings, especially when results are needed urgently and as long as doctors advise patients about the potential false negatives. And patients, indeed everyone, still need to be counseled to continue taking precautions to avoid infecting other people, he says.
"It is a risk that if you tell somebody they're negative and they're truly positive that they will relax social distancing, not wear a face mask, etc., and could transmit the disease," Procop says.
"If you get back a negative test and think, 'I don't have it and I can go to a mixer,' " Procop says. "And now everybody at the mixer has now been exposed to you."
"A high rate of false negatives would definitely be cause for concern," says Dr. Thomas Inglesby, who runs the Center for Health Security at the Johns Hopkins School of Public Health.
"If that happens, then they wouldn't be told to isolate themselves, public health wouldn't seek out or quarantine their close contacts, doctors wouldn't know what illness they have. Their family members wouldn't know to protect themselves," Inglesby wrote in an email.
Other researchers noted that all tests can produce false-negative results, especially if the samples are not collected properly or at a time when a patient doesn't have readily detectable levels of virus in their bodies.
"All of the tests do miss a number of patients — anywhere from 5 to 30%," says Dr. Alan Wells, a professor of pathology at the University of Pittsburgh. "So this is not unique to ID NOW."
And just because someone tests negative one day, doesn't mean they might not get infected the next day.
"So regardless of whether it's ID NOW or other tests, we recognize that we cannot say with any assuredness that you are safe or not," Wells says.
But Wells added that "the ID NOW test seems not to be as effective as" other tests, based on his own research.
In Wells' study involving specimens from 50 patients, the ID NOW missed more infections than three other tests, especially in samples containing the lowest levels of virus, Wells says.
"The ID NOW missed roughly about half those patients," Wells told NPR in an interview.
"We do not use ID NOW to rule out a diagnosis of COVID disease, based on the limits of it not picking up as many patients as other platforms that we have," Wells said. "Our use of ID NOW is put on hold."
Wells also doubts the false-negatives are due to the specimens being diluted by being stored in fluid first.
"It's not due to the dilution," Wells said. "Our evidence is that this is not the sole cause of the lower effectiveness of the ID NOW."
NOEL KING, HOST:
Three states in the South are planning to reopen parts of their economies in the next few days. The governors of Georgia, South Carolina and Tennessee have said they're going to ease restrictions. Georgia's governor, Brian Kemp, says his state has upped testing and that health professionals have given the go ahead. Georgia gyms, bowling alleys and salons will be able to open as long as they follow social distancing and cleaning guidelines. Now, these reopenings are happening despite the fact that there's still a lot of concern over testing shortages in the United States. And there is another problem. NPR has learned that some new, faster tests may not be as reliable as hoped. Health correspondent Rob Stein broke this story.
ROB STEIN, BYLINE: Hey, there. Good morning.
KING: Tell me about this new, fast test and what's going on.
STEIN: It's called the Abbott ID NOW test, and it's gotten a lot of attention because it's so fast. It can tell someone if they're negative or positive in less than 15 minutes, you know, like the quick flu or strep test we're used to seeing clinics and doctors' offices. You know, President Trump has bragged about this a lot at the White House briefings. And lots of people have been talking about how the Abbott ID NOW tests could help life get back to normal.
KING: OK, but there is a problem with it. What's the problem?
STEIN: Well, you know, it appears this test can miss more infected people than other tests. Dr. Gary Procop is a top medical testing expert at the Cleveland Clinic. He shared with NPR the results of what could be the biggest study so far. Take a look at this. He tested 239 specimens with the ID NOW test and four other commonly used tests. Here's what he found.
GARY PROCOP: The Abbott test in our five-test comparison produced the most false negatives out of all the tests.
STEIN: He says that test detected only 85.2% of the positive specimens.
PROCOP: So that means, you know, if you had 100 patients that were positive, 15% of those patients would be falsely called negative. They'd be told that there are negative for COVID when they're really positive.
KING: Wow. So 15% of people could be walking around thinking that they are in the clear when in fact they're sick. They're positive.
STEIN: Yeah, yeah. That's what he's saying. And, you know, the concern is they could spread it to other people. Now, this test isn't the most common test out there today, but 50,000 are being produced each day. And that's increasing fast. So for one thing, Procop says that these false negatives make it really dangerous for deciding whether it's safe, for example, you know, to have patients come into hospitals maybe for, like, elective surgeries. Let's listen to a little bit more of what he said about that.
PROCOP: You know, if you have a patient coming in the hospital and you're going to put them into what has been determined to be a COVID-free ward, you have to have the most sensitive tests available. Because once you put somebody with COVID into a COVID-free ward, it's no longer a COVID-free ward anymore. It's your new COVID ward.
STEIN: You know, so Procop says his hospital stopped using the test that way. Instead, they're going with the three other tests that hits the mark between 96% and 100% of the time.
KING: So what's happening with this test now? Is it being used for other things?
STEIN: Yeah. You know, it's being used to test doctors and nurses, you know, regular people who are worried they may be sick, family members of sick people. And there's talk about using it to help decide all sorts of things, like, you know, who's safe to go back to work or get on a plane. Here's Gary Procop again from the Cleveland Clinic.
PROCOP: It is a risk that if you tell somebody they're negative and they're truly positive that they will relax social distancing, not wear a face mask, et cetera and could transmit the disease. So that's a truth.
KING: Rob, let me ask you about Abbott, the company that makes the test. What are they saying about this?
STEIN: You know, Abbott says the test is very reliable and any problems are not being caused by the test itself. It could be how it's being used, that specimens are being diluted instead of going directly into the Abbott machines like they're supposed to. But Procop says the company needs to prove that. Now, you know, it's important to note that all tests can produce false negatives, you know, if the sample isn't collected the right way or at the right time. And just because someone tests negative one day, they may get infected the next day. And the Abbott ID NOW quick test can be very useful, especially when it's really important to get results really fast. And most of the time, it is right. But Procop says it's important to know that even when someone gets a negative test, they may still be infected. You know, they can't get lulled into a false sense of security. They can't let their guard down.
KING: Health correspondent Rob Stein. Thanks so much for this reporting, Rob.
STEIN: You bet. Transcript provided by NPR, Copyright NPR.