Opinion: Dentistry, Diagnostic Test Accuracy (DTA) and the Covid-19 Antibody Test

Link to the Dental Elf

Virus Outbreak Germany

As we are acutely aware the whole country has been locked down in an attemp to reduce the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2 (Covid-19)), while we restructure the healthcare systems and direct research funding into either a treatment and/or a vaccine. The next stage is to rapidly scale up diagnostic testing to collect robust base-rate data at both an individual, and population level for future high quality decision-making (Ioannidis, 2020). The need to know who has been infected will be particularily important for dental care professionals due to the potential risk of Covid-19 spread via aerosol generating procedures (Coulthard, 2020). So what is the current state of affairs regarding quick point-of-care antibody testing? How good is it, how good should it be, and why is this important?

Firstly I am going to use a recently published paper that evaluated 10 antibody tests for SARS-CoV-2 using Enzyme-Linked Immunosorbent Assay (ELISA) and Lateral Flow Immune Assay (LFIA), lateral flow assay utilise the same technology commonly used for pregnancy tests (Crook, 2020). This particular paper has sparked considerable controversy which I will go into later. If you need more detail on how these tests work and their advantages/disadvantages then please take a look at the Oxford COVID-19 Evidence Service (Green et al., 2020). When looking at diagnostic accuracy testing the two main concepts to understand are:

Sensitivity The ability of a diagnostic test to give a positive result when it is supposed to be positive.

Specificity The ability of a diagnostic test to indicate a negative result when it is supposed to be negative.

 Results

After extracting the primary data from the individual tests, meta-analysis was carried out using the ‘mada’ package in R. The summary estimate for sensitivity was 62.7% (95%CI: 57.5 to 67.7) and specificity 96.9% (95%CI: 95.2 to 98.0) The results for the ELISA test and the 9 LFIA tests were plotted below on to a Summary Receiver Operating Characteristic (sROC) curve ( See Figure 1). The y-axis represents the sensitivity (1.0 =100%), and the x-axis represents 1- specificity (0.10 = 10%). For a test to be perfect the summary estimate point should be in the top left corner representing 100% true-positives and 0% false-positives. The blue dot represents the summary estimate for the tests surrounded by a 95% confidence area, the red dot represents the specification target of >98% (95%CI: 96 to 100%)  for sensitivity, and >98% (95%CI: 96 to 100%) for specificity, set by the Medicines & Healthcare Products Regulatory Agency (MHRA., 2020).

Figure 1. sROC curve – Antibody tests

sROC_DT_1_LI (2)

Discussion

So why is so important to set the levels of sensitivity and specificity so high? Imagine we have a small city with a population of 100,000. The prevalence of people who have had the virus and recovered is 6%  and you are using an LFIA test with a sensitivity of 63% and specificity of 97%. Out of 6600 people who test positive for virus antibody only 3780 are true positives which corresponds to 57% having a correct positive diagnosis (See Figure 2.). Another important consideration is that 3% of the population who do not have antibodies test positive (n= 2820) and could lead them to believe they are immune.

Figure 2.Frequency tree of Covid-19 antibody screening

Annotation 2020-05-09 193854

The second point about this paper is that in its present format it cannot be used in any future analysis since the companies that supplied the tests required a commercial confidentiality agreement to be signed with the UK Department of Health making it impossible to discriminate between tests. The current set of results show poor performance, and that is why the MHRA has specifically set its targets high because of the risks that  results could pose if they were used to ease a lockdown, or they become part of an immunity passport system (Mahase., 2020; WHO, 2020). The final point is that whenever we have to deal with diagnostic tests or screening devices in either our professional or private lives we need to be able to identify the products, their comparators, and how accurate they are before making a significant decision to use or purchase the product. False-positive and false-negative results can pose significant harms to both ourselves and the population in general.

Disclaimer:  The article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author and not necessarily those of the host institution. The views are not a substitute for professional advice.

References

COULTHARD, P. 2020. Dentistry and coronavirus (COVID-19) – moral decision-making. Br Dent J, 228, 503-505.

CROOK, D. W. 2020. Evaluation of antibody testing for SARS-CoV-2 using ELISA and lateral flow immunoassays [Online]. Department of Microbiology, John Radcliffe Hospital, Oxford, OX3 9DU, United Kingdom.  [Accessed].

GREEN, K., , A. W., DICKINSON, R., GRAZIADIO, S., ROBERT WOLFF, MALLETT, S. & ALLEN, A. J. 2020. What tests could potentially be used for the screening, diagnosis and monitoring of COVID-19 and what are their advantages and disadvantages? [Online]. Available: https://www.cebm.net/covid-19/what-tests-could-potentially-be-used-for-the-screening-diagnosis-and-monitoring-of-covid-19-and-what-are-their-advantages-and-disadvantages/ [Accessed].

IOANNIDIS, J. P. 2020. A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data [Online]. STAT. Available: https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/ [Accessed].

MAHASE., E. 2020. Covid-19: Confidentiality agreements allow antibody test manufacturers to withhold evaluation results.

MHRA.2020.Target_Product_Profile_antibody_tests_to_help_determine_if_people_have_immunity_to_SARS-CoV-2_ [Online]. Available: https://www.gov.uk/guidance/guidance-on-coronavirus-covid-19-tests-and-testing-kits [Accessed].

WHO.2020. “Immunity passports” in the context of COVID-19 [Online]. Available: https://www.who.int/news-room/commentaries/detail/immunity-passports-in-the-context-of-covid-19 [Accessed].

Disclaimer: the article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author. The views are not a substitute for professional medical advice.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s