Translations: "Nederlands" |

Why the number of positive tests will never reach 0 (Corona blog part 2)

There has been a lot of discussion about the so-called polymerase chain reaction (PCR) test lately. This highly advanced test, if used properly, may be a powerful tool in the control of the SARS-COV-2 virus. It is important, however, that the results are handled correctly.

A positive PCR test does not necessarily mean you are sick!

Attentive readers of the previous blog will have noticed that I carefully avoided the words infections (let alone 'contagious people', as our minister Hugo de Jonge keeps blurting out) and instead used the term 'positive test result'. This is because a positive test result does not immediately mean that you have the virus or that you are, or will become, contagious. A number of arguments why you should take such a result with a grain of salt are:

• There is a debate whether the Corman-Drosten paper on which the PCR test is based is scientifically sound. There is even an official retraction paper that could lead to the retraction of the Corman-Drosten paper which then loses scientific substantiation. What the test still says if that happens is uncertain. Maybe we are measuring the wrong things entirely.
• Having (parts of) the virus RNA in your mucus is not automatically a sign of infection. They could be fragments of a previous infection or just parts of the virus that never got past your first line of defense.
• There is a significant chance of a so-called 'false positive' result or a positive result while you do not have the virus with you. There is an even greater chance of a false negative result, which means that the test does not detect the virus while you do have it.

I will not comment on the first two points here. In the meantime I know quite a bit about it and I read these things with interest, but it is not my expertise. The third point lies within the domain of probability calculation and statistics and so with my knowledge and experience I can say something about that.

Sensitivity and specificity

This brings us to the subject of sensitivity and specificity. These two, quite difficult, concepts are basic knowledge for every GP and are very important in the evaluation of any (not just the PCR) test. No test is 100% accurate. There is always a chance of mistakes. We therefore always distinguish two important values in medical tests that indicate the reliability of a test:

• Sensitivity: The chance of a 'positive' result while you are carrying the virus.
• Specificity: The chance of a 'negative' result while you are not carrying the virus.

A good test has both a high sensitivity and a high specificity. Specificity, in particular, is critical when testing large groups, as we'll see in a moment.

For most tests the values for the sensitivity and specificity are known with reasonable accuracy. This is different for the PCR test designed for SARS-COV-2. This is partly because the test has not been around for very long, has been designed quite hastily and not enough time has been taken for validation. It is difficult to find reliable figures and the values that I have found vary greatly per study. As far as I know, based on the current research for the PCR test used in the Netherlands, you can state that:

• The sensitivity is somewhere between 67% and 98%.
• The specificity is somewhere between 96% and 99.8%.

The figures mentioned come from literature research of the RIVM itself and I have little reason to doubt these numbers. These are very wide margins and you may rightly ask yourself whether the test can be used for 'screening' large groups of people. We will see that this strongly depends on the situation.

It is important to realize that these estimates are based on laboratory conditions. Tests taken in a hastily built test line by hastily trained personnel where the samples are transported outside the laboratory and there is time pressure to present the results by different laboratories with different measuring equipment are far from comparable to laboratory conditions but this is what we have and so this is what we are going to use to set up a calculation example. We assume a value of 80% for the sensitivity and a value of 98% for the specificity. About in the middle of the range that emerges from the studies.

Suppose as an example that we would like to test everyone in the Netherlands and quarantine those people with a positive result? Well, that can be calculated: At the time of writing, RIVM estimates the number of infected people in the Netherlands to be roughly 0.5% of the Dutch population. The contamination level is therefore 0.5%. Then the picture looks like this with a population of 17 million people:

• 80% x 0.5% x 17 mln = 68,000 people are rightly told they have the virus and must be quarantined.
• (100-80)% x 0.5% x 17 mln = 17,000 people get false negative results and therefore are erroneously excluded from quarantine.
• (100-98)% x 17 mln = 340,000 people get a false positive result and are therefore unjustly quarantined.

For an individual in this test this means that if he or she has tested positive there is a chance of as much as 340 / (68 + 340) x 100% = 83% that this is an incorrect test result. 83 out of 100 people are unfairly quarantined! Maybe a viable option in a country like China, but (hopefully, I don't rule out anything anymore in these strange times) not in the Netherlands! And then we have not even eradicated the virus because we miss 17,000 people who continue to walk around with the virus and infect others again.

The situation I outline above is completely counterintuitive to most people. It is therefore known as the False Positive Paradox and is basic knowledge for general practitioners, among others. Screening the whole of the Netherlands (or any other large group of people without indication) with this test is therefore a very bad idea. The situation changes when the infection rate of the disease being studied is much higher. I have briefly made an overview of the chance of a false positive result (FP) with an infection rate of 0.5% and 10% respectively:

At a contamination level of 0.5% it does not look good. Best case, a third of the people are wrongly in quarantine. Worst case 91%. A 9 out of 10 chance that you can sit at home while nothing is wrong! Completely unacceptable! And this is precisely the reason that RIVM would like only people with complaints to be tested. The hope is that the percentage of infections with SARS-COV-2 in this group will be much higher than 0.5%, so that the influence of specificity is less important on the results. And indeed, we see a percentage of around 10% positive. With a specificity of 98%, according to our table, you have 'only' a 20% chance of being unfairly at home.

It is therefore very unwise to test people without complaints or other indications given the current level of infection. The effects can be guessed in advance: The relative contamination figures will then decrease while the percentage of false positives increases. The result is that a relatively large number of people will have to go into quarantine unfairly, while your detection rate will hardly improve (or possibly even worsen).

Retesting is not a solution

Although it is true that if you have tested positive, you could rule out that you are wrongly in quarantine with a new test, the guidelines of the RIVM prohibit this. That feels strange, but that restriction is there for a good reason; Due to the relatively low selectivity of the test, there is a good chance that someone carrying the virus will get a negative result in a retest. If we retested everyone who tested positive, then with a selectivity of 80% instead of 20%, around 36% of the infected people would not have to be quarantined. In fact, at a selectivity of 70%, slightly more than half of infected people would avoid quarantine. So with this test it is a choice between a rock and a hard place: Falsely placing a significant number of people in quarantine in order to be able to keep as many actually infected people in quarantine as possible or to keep as few people as possible in quarantine and missing a large proportion of the infected people. A difficult decision indeed!

Never back to normal again

There is another danger in testing large numbers of people with such a test: Suppose the virus has left the Netherlands and no one is infected anymore. And suppose we test 100,000 people daily with a PCR test with a specificity of 98%. As long as we do that, there will be 2000 people who will come out positive every day. The number will never become lower as that is simply the inaccuracy of the test. There is no way to find out if people are still infected and if so who they are. If we stick to that, we can never go back to normal simply because it seems like the virus never goes away. The 'reproduction number' R will always hover around 1 and it will seem like we never get the virus under control. An everlasting threat that keeps us living in fear forever. The farther the virus is gone, the closer we drift to this point. We may even have unknowingly reached it last summer! But since we don't know the specificity of the test at all, how do we know when that point has been reached? At some point we will simply have to stop testing. Hopefully politics will eventually see that.

Pointers for improvement

A number of things are not going well in my opinion; Politicians and media must first of all realize that at very low contamination levels, such as last summer, the test has hardly any value. Scaling up the tests will then automatically lead to more false positives and thus to unnecessary anxiety. Actually, you should no longer screen with this test when the contamination level of the group to be tested falls below 3%. Below that, you probably have more false positives than valid results. I would therefore advise scaling down the number of tests in the test lanes when the contamination rate decreases and eventually reducing it to 0. Further testing can then take place via the normal route: first to the doctor who can, if necessary, have a test performed. In this way you take the fear and unrest out of society while you can still keep an eye on the development of the virus.

Conclusion

The current PCR test is actually not suitable for screening large groups at a low level of contamination. Even at the current rate of infection, there is a considerable chance (estimated at 20%) that you will not carry the virus at all, even with a positive test. Last the summer that chance was probably more than 50%. Caution is therefore advised in screening large groups using the PCR test and the resulting figures should be approached with great caution. There are major risks with the current testing policy especially when policymakers, who often do not understand enough of the mechanisms that play a role here, start linking conclusions and measures to the measurement results. Further research into the selectivity and specificity of the PCR test (especially when practically applied in test lanes, not under laboratory conditions) remains important because it gives us a better picture of the reliability of the test and allows better coordination of test policy.