Re-opening of dental services: A rapid review of international sources. Part II.

Separating the signal from the noise regarding masks.

Link to the Dental Elf

File:Face Masks used to prevent the spread of Coronavirus in ...

Bottom Line

As the number of clinical guidelines and standard operating procedures increases, we are seeing a reduction in consensus regarding a clear way forward in patient management. If we are going to take an evidence-based approach in a land devoid of direct evidence the policy makers are going to have to defer to the clinical expertise, and local knowledge of the profession in regards to the choice of PPE regarding Covid-19 negative patients requiring aerosol generating procedures.

Background

The recent ‘Cochrane Recommendations for the re-opening of dental services: a rapid review of international sources’ has been updated  on the 16th May to include a further 5 international guidelines. The document now reviews 17 guidance documents from 16 countries (France, Spain, Portugal, Austria, Switzerland, Belgium, Netherlands, Norway, Denmark, Malta, America CDC, America ADA, Canada, Australia, New Zealand, India). The common themes and the relevant recommendations were divided into 5 domains:

  1. Practice preparation and patient considerations.
  2. Personal protective equipment (PPE) for dental practice personnel.
  3. Management of the clinical room.
  4. Dental procedures.
  5. Post-operative cleaning/disinfection/waste management.

Methods

The data was extracted in a similar way to the previous review. The five domains have mostly remained the same with little change in domains 1, 3, 4, and 5. The exception being domain 2 (PPE for dental practice personnel). The domain almost doubled in size to 29 subgroups from the original 15, but at the same time there was a reduction in the consensus with the mean dropping from 58% to 30% (See Figure 1.).

Update_f1

As with the previous review I selected the subgroups that achieved to filter out the large number of subgroups with a low degree of consensus. In the original paper there were 10 subgroups that scored ≥ 50%, in the update this reduced to 7 which constitutes a 42% reduction in agreement considering the expansion of the domain. (See Figure 2).

Figure 2. Subgroups with greater than 50% consensus.

Update_f2

 

Looking at these subgroup headings there is a lot of duplication especially regarding the use of FFP2 and FFP3 masks and the infective status of the patient. To try and clarify this matter a subgroup analysis was undertaken (See Figure 3,).

Figure 3. Mask selection

update_f3

From this chart it was obvious that for Covid negative patients the consensus was for the use of surgical masks, and with Covid infective patients requiring an aerosol generating procedure (AGP) an FFP3 mask should be used. There was however a grey area around the use of a surgical mask combined with a face shield/visor, and an FFP2 mask for Covid negative patients requiring an AGP, the prevalence of asymptomatic patients being very low in the community (ONS, 2020). There was 41% (95% CI: 18% to 65%) agreement regarding the surgical masks, against 59% (95% CI: 35% to 82%) agreement for the use of an FFP2 mask, however as the sample size was small (n=17) the result was not statistically significant (p= 0.29).

Discussion

What can we conclude from the update? Due to the lack of hard evidence regarding the effectiveness of PPE in the real-world clinical environment the guidelines would appear to be based on opinion that ranges from the precautionary principle that all patients should be considered infective, to a more pragmatic approach based on the professions current cross infection strategies. In a perfect world we would like to see well  design randomised controlled studies to answer these questions, but this does not address the here-and-now of real-world dentistry. If we are going to follow the principles of evidence-based dentistry (Sackett et al., 1996) the clinicians are going have to make the decision of using surgical masks, or FFP2 masks  for AGPs based on individual clinical expertise, best available evidence, and their patients values and preferences rather than rigid guidelines that can’t adapt to local circumstances.

References

ONS. 2020. Coronavirus (COVID-19) Infection Survey pilot: England, 14 May 2020 [Online]. Available: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/england14may2020 [Accessed].

SACKETT, D. L., ROSENBERG, W. M., GRAY, J. M., HAYNES, R. B. & RICHARDSON, W. S. 1996. Evidence based medicine: what it is and what it isn’t. British Medical Journal Publishing Group.

https://oralhealth.cochrane.org/news/recommendations-re-opening-dental-services-rapid-review-international-sources

Are we sleepwalking into PPE paralysis?

“When the facts change, I change my mind. What do you do, Sir?” – John Maynard Keynes

sleepwalkers-figure-roof-sky-royalty-free-thumbnail

Link to Dental Elf

Bottom line answer

As we approach a return to clinical practice policy makers need to be mindful that the dental profession is already highly proficient in cross infection control, and the benefits of new elaborate PPE protocols regarding aerosol generating procedures may be marginal in the light of low disease prevalence. If we need 300,000 participants in a  study it may be impossible to practically demonstrate  significant benefits to patient safety from the perfect PPE model compared to the harms created by expense and access.

 

Background

The most important element of Keynes’s famous quote relating to the current coronavirus pandemic is the word ‘fact’. Facts at this moment in time are constantly changing as this disease spreads, and what was true two weeks ago is now a distant memory. One major problem we face as far as healthcare policy is concerned is a lack of accurate base-rate data on the prevalence of the disease in the population, this was initially addresses by Professor Ioannidis (Ioannidis, 2020). His article in STAT caused quite a lot of online controversy (Bastian, 2020, Taleb, 2020) but what it did is highlight the difficulties of being objective and human. At this moment in time it is extremely difficult not to be influenced by the problems of base-rate neglect, loss aversion, and availability bias via the media as we count the daily international infection/death figures (Gaurav, 2020). In this post I want to concentrate on how important it is to understand the significance of accurate base-rate (prevalence) reporting  so we can allocate the correct amount of training and resources to the dental profession based of the potential aerosol risk in virus transmission. In a paper by Chambers on base-rates in dental decision-making there is a quote (Chambers, 1999):

‘Base your decisions on either the baseline alone or the evidence alone, depending on which one contains the most information.’

What we are seeing now is a rapid accumulation of both base-line data and evidence,  but policy decisions about the future are being based on data and precautionary principles that were only valid at the start of this pandemic. To highlight this, I would like to explore how we are going to test the real-world effectiveness of the personal protective equipment (PPE) and cross infection protocols that are flooding the profession now.

Methods

I am going to look at three areas here, base-rates (prevalence), numbers of asymptomatic individuals, and power calculations regarding PPE use. For clarity I will use natural frequencies wherever possible.

Firstly, on the 14th May the Office of National Statistics (ONS) in the UK published the results of its coronavirus (Covid-19) infection survey (ONS, 2020). This data was based on 10,705 participants’ swab tests taken over a two-week period from 27th April to 10th May, the sample was drawn from households in which someone has already participated in an ONS survey to ensure the sample was representative of the wider population. From this sample 33 individuals in 30 households tested positive for COVID-19. This equates according to the ONS to 0.27% (95% confidence interval: 0.17% to 0.41%) of the population of England.

Secondly, one of the key problems with Covid-19 is asymptomatic spread (Bai et al., 2020) but there is no reliable data so any calculations here need to be looked on as a Fermi (back of an envelope) problem. To get a best estimate on the proportion of asymptomatic patients I conducted a meta-analysis of the data presented on the Oxford Covid-19 Evidence Service (Heneghan et al., 2020). The meta-analysis was carries out in R using a random effects model (See Figure 1.)

Figure 1. Forest plot of asymptomatic individuals

Asympto

There are two points of note from this forest plot, the summary estimate for asymptomatic individuals is 27% (95% CI: 12 to 45%) and the heterogeneity between studies (variability) is extremely high.

The next stage is to put the base-rate and number of asymptomatics together in the form of a frequency tree. For ease of calculation I have rounded the figures so a base-rate of 0.27% becomes 1 in 400,  and the number of asymptomatics becomes 30% (See Figure 2.).

Figure 2. Frequency tree of asymptomatic vs symptomatic

Asympto_1

The frequency tree illustrates that in this population 1 out of every 1333 people could be an asymptomatic carrier of Covid-19.

How does this relate to dentistry? On the precautionary principle we are operating under at the moment the presumption is that all (100%) the patients are asymptomatic carriers rather than the true figure of 0.075% ( I have assumed symptomatic patients will not be attending a dental surgery or will be triaged out prior to entering the clinical environment). This becomes important when we want to test if our PPE and protocols are effectively protecting both the patients and the staff.  We now need to set up a study comparing PPE that is adequately powered to eliminate the effects of random chance around such a small prevalence statistic (Button et al., 2013). I have created three examples of PPE for aerosol generating procedures:

  • Perfect PPE model (fluid resistant disposable gowns, FFP3 masks, visors, ventilation, long fallow periods etc) with a 99%  chance of reducing viral contamination
  • Realistic expectations of enhanced PPE practice (FFP2, reusable surgical gown, rubber dam etc) at 93%.
  • Standard practice (surgical masks etc) at 80%.

I placed the data into an apriori sample size calculator (G*Power 3.19.2) with a error probability is 0.05 and power (1-b error probability) of 0.8 (See Table 1).

Table 1. Sample sizes for a well powered study into PPE effectiveness

asympto_tableAs we can see even in a simulation study, we are going to have to at least place over a hundred individuals in each arm of the study. To see if the benefit translates into the real-world, we need to go up two orders of magnitude to see if there is a significant difference between perfect and good PPE  based on accurate population base-rate figures.

Discussion

The purpose of this opinion paper was to highlight the potential problems that a precautionary principle can create in healthcare when we work on the assumption that 100% of the patients attending a dental surgery are infectious. Guidelines and protocols need to take into consideration the absolute risk within the population based on data that is accurate and up to date. Simulation studies, and pilot studies rarely carry their full reported success into the real world (Kistin and Silverstein, 2015). Without taking a deep breath and objectively assessing the changing data regarding Covid-19, policy makers, academics, and clinicians can unconsciously fall fowl of the base-rate fallacy and availability biases created by the modern media. High quality PPE and staff training is a vital component of keeping everyone safe from this virus but we must be mindful of the other effects that perfect practice can have on the health economics and affordability of health care to those most vulnerable.

Disclaimer:  The article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author and not necessarily those of the host institution. The views are not a substitute for professional advice.

References

BAI, Y., YAO, L., WEI, T., TIAN, F., JIN, D. Y., CHEN, L. & WANG, M. 2020. Presumed Asymptomatic Carrier Transmission of COVID-19. JAMA.

BASTIAN, H. 2020. A rebuttal to “A fiasco in the making?” [Online]. Available: http://hildabastian.net/index.php/8-secondary/87-a-rebuttal-to-a-fiasco-in-the-making [Accessed].

BUTTON, K. S., IOANNIDIS, J. P., MOKRYSZ, C., NOSEK, B. A., FLINT, J., ROBINSON, E. S. & MUNAFÒ, M. R. 2013. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376.

CHAMBERS, D. W. 1999. The roles of evidence and the baseline in dental decision making. J Am Coll Dent, 66, 60-7.

GAURAV, S. 2020. Behavioural Economics in the Fight Against COVID-19: BOMA Framework.

HENEGHAN, C., BRASSEY, C. & JEFFERSON, T. 2020. COVID-19: What proportion are asymptomatic? [Online]. Available: https://www.cebm.net/covid-19/covid-19-what-proportion-are-asymptomatic/ [Accessed].

IOANNIDIS, J. P. 2020. A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data [Online]. STAT. Available: https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/ [Accessed].

KISTIN, C. & SILVERSTEIN, M. 2015. Pilot studies: a critical but potentially misused component of interventional research. Jama, 314, 1561-1562.

ONS. 2020. Coronavirus (COVID-19) Infection Survey pilot: England, 14 May 2020 [Online]. Available: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/england14may2020 [Accessed].

TALEB, N. 2020. EVIDENCE BASED is often BS [Online]. Available: https://twitter.com/nntaleb/status/1240641133820207104 [Accessed].

 

Opinion: Dentistry, Diagnostic Test Accuracy (DTA) and the Covid-19 Antibody Test

Link to the Dental Elf

Virus Outbreak Germany

As we are acutely aware the whole country has been locked down in an attemp to reduce the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2 (Covid-19)), while we restructure the healthcare systems and direct research funding into either a treatment and/or a vaccine. The next stage is to rapidly scale up diagnostic testing to collect robust base-rate data at both an individual, and population level for future high quality decision-making (Ioannidis, 2020). The need to know who has been infected will be particularily important for dental care professionals due to the potential risk of Covid-19 spread via aerosol generating procedures (Coulthard, 2020). So what is the current state of affairs regarding quick point-of-care antibody testing? How good is it, how good should it be, and why is this important?

Firstly I am going to use a recently published paper that evaluated 10 antibody tests for SARS-CoV-2 using Enzyme-Linked Immunosorbent Assay (ELISA) and Lateral Flow Immune Assay (LFIA), lateral flow assay utilise the same technology commonly used for pregnancy tests (Crook, 2020). This particular paper has sparked considerable controversy which I will go into later. If you need more detail on how these tests work and their advantages/disadvantages then please take a look at the Oxford COVID-19 Evidence Service (Green et al., 2020). When looking at diagnostic accuracy testing the two main concepts to understand are:

Sensitivity The ability of a diagnostic test to give a positive result when it is supposed to be positive.

Specificity The ability of a diagnostic test to indicate a negative result when it is supposed to be negative.

 Results

After extracting the primary data from the individual tests, meta-analysis was carried out using the ‘mada’ package in R. The summary estimate for sensitivity was 62.7% (95%CI: 57.5 to 67.7) and specificity 96.9% (95%CI: 95.2 to 98.0) The results for the ELISA test and the 9 LFIA tests were plotted below on to a Summary Receiver Operating Characteristic (sROC) curve ( See Figure 1). The y-axis represents the sensitivity (1.0 =100%), and the x-axis represents 1- specificity (0.10 = 10%). For a test to be perfect the summary estimate point should be in the top left corner representing 100% true-positives and 0% false-positives. The blue dot represents the summary estimate for the tests surrounded by a 95% confidence area, the red dot represents the specification target of >98% (95%CI: 96 to 100%)  for sensitivity, and >98% (95%CI: 96 to 100%) for specificity, set by the Medicines & Healthcare Products Regulatory Agency (MHRA., 2020).

Figure 1. sROC curve – Antibody tests

sROC_DT_1_LI (2)

Discussion

So why is so important to set the levels of sensitivity and specificity so high? Imagine we have a small city with a population of 100,000. The prevalence of people who have had the virus and recovered is 6%  and you are using an LFIA test with a sensitivity of 63% and specificity of 97%. Out of 6600 people who test positive for virus antibody only 3780 are true positives which corresponds to 57% having a correct positive diagnosis (See Figure 2.). Another important consideration is that 3% of the population who do not have antibodies test positive (n= 2820) and could lead them to believe they are immune.

Figure 2.Frequency tree of Covid-19 antibody screening

Annotation 2020-05-09 193854

The second point about this paper is that in its present format it cannot be used in any future analysis since the companies that supplied the tests required a commercial confidentiality agreement to be signed with the UK Department of Health making it impossible to discriminate between tests. The current set of results show poor performance, and that is why the MHRA has specifically set its targets high because of the risks that  results could pose if they were used to ease a lockdown, or they become part of an immunity passport system (Mahase., 2020; WHO, 2020). The final point is that whenever we have to deal with diagnostic tests or screening devices in either our professional or private lives we need to be able to identify the products, their comparators, and how accurate they are before making a significant decision to use or purchase the product. False-positive and false-negative results can pose significant harms to both ourselves and the population in general.

Disclaimer:  The article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author and not necessarily those of the host institution. The views are not a substitute for professional advice.

References

COULTHARD, P. 2020. Dentistry and coronavirus (COVID-19) – moral decision-making. Br Dent J, 228, 503-505.

CROOK, D. W. 2020. Evaluation of antibody testing for SARS-CoV-2 using ELISA and lateral flow immunoassays [Online]. Department of Microbiology, John Radcliffe Hospital, Oxford, OX3 9DU, United Kingdom.  [Accessed].

GREEN, K., , A. W., DICKINSON, R., GRAZIADIO, S., ROBERT WOLFF, MALLETT, S. & ALLEN, A. J. 2020. What tests could potentially be used for the screening, diagnosis and monitoring of COVID-19 and what are their advantages and disadvantages? [Online]. Available: https://www.cebm.net/covid-19/what-tests-could-potentially-be-used-for-the-screening-diagnosis-and-monitoring-of-covid-19-and-what-are-their-advantages-and-disadvantages/ [Accessed].

IOANNIDIS, J. P. 2020. A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data [Online]. STAT. Available: https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/ [Accessed].

MAHASE., E. 2020. Covid-19: Confidentiality agreements allow antibody test manufacturers to withhold evaluation results.

MHRA.2020.Target_Product_Profile_antibody_tests_to_help_determine_if_people_have_immunity_to_SARS-CoV-2_ [Online]. Available: https://www.gov.uk/guidance/guidance-on-coronavirus-covid-19-tests-and-testing-kits [Accessed].

WHO.2020. “Immunity passports” in the context of COVID-19 [Online]. Available: https://www.who.int/news-room/commentaries/detail/immunity-passports-in-the-context-of-covid-19 [Accessed].

Disclaimer: the article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author. The views are not a substitute for professional medical advice.