The Tyranny of Average and Guideline Development for PPE

‘All models are wrong, but some are useful’ George Box

Link to The Dental Elf

3D_medical_animation_corona_virus

The Bottom Line

It is important when developing national and local guidelines to appreciate the extremely high variability  in clinical risk that Covid-19 poses to the dental team, and the patients they serve. Regarding the use of PPE and aerosol generating procedures there should be sufficient flexibility in the national guidelines to allow clinicians to safely apply their local knowledge so that the PPE selected matches the degree of risk for the individuals. Applying a rigid, unsustainable PPE policy will waste valuable resources while increasing potential harm to staff and patients.

Background

During the Q&A section of a webinar I was recently presenting about evidence-based dentistry one of the participants made a very interesting point. As a surgeon working in the front line of the Covid-19 pandemic not all areas have been effected the same, in some areas of the country we have hardly seen any deaths in the community, and in a North London hospital the staff there are seeing many severe cases a day. This point that the infection is not evenly distributed either geographically nor demographically is important, as it highlights the ‘tyranny of the average’(Merlo et al., 2017) in terms of healthcare policy and guideline development. From the data this virus is not an ‘equal opportunities killer’ as the highest mortality rates  are effecting the frail, disadvantaged, and highlighting strong racial/ethnic inequalities (Webb et al., 2020, Everson et al., 2002). A clear example of this is that out of 23,804 Covid-19 related deaths 31.4% (7466) occurred in people with Type II diabetes (prevalence in pop. approx. 6%). 15% of  Covid-19 admissions  to hospital had no reported medical conditions, but 50% had three or more significant comorbidities (ISARIC, 2020) (See Figure 1.).

Figure 1.The distribution of combinations of the four most common chronic diseases (ISARIC 2020)

Comorbidities

An example of this variation can be seen in regional mortality figures for England. Mortality with a diagnosis of Covid-19 is the only reliable endpoint measurement  until we have large scale reliable track and trace/antibody tests. From the background information we can deduce that there appears to be significant variation between geographic locations with lower risk in the more rural counties and smaller cities by as much as a factor of 2.5.(Table 1).

Table 1. Differences in regional mortality up to 8th May

Location Population Population density/km2 Covid-19 deaths % of population die of Covid-19 Risk Ratio
Plymouth 262100 8500 57 0.021 1
Lincolnshire 761224 127 207 0.027 1.29
Manchester 552858 4716 289 0.052 2.48
Tower Hamlets 324745 16000 172 0.053 2.52

Methods

The following calculations are a Fermi problem after Enrico Fermi, where an estimate is made with little or no data and used to identify orders of magnitude rather than point estimates. The methodology undertaken uses the mathematical principles laid out by Pólya in his famous book ‘How to Solve it’ (Polya, 1945). These are

  • Understand the problem.
  • Find the connection between the data and the unknown and devise a plan.
  • Run the model.
  • Examine the solution.

In a previous post I looked at base rates as a point estimate by using the mean without a confidence interval (Howe, 2020). Since then two reviews have been published  investigating the infection fatality rate (IFR) (Ioannidis, 2020, Meyerowitz-Katz and Merone, 2020). The IFR estimates the overall mortality rather than the number of deaths as a proportion of confirmed cases (CFR). Due to variations in analysis the two reviews produce significantly different results, Ioannidis’s review produces as IFR range of 0.02% to 0.40%, and the Meyerowitz-Katz review a mean of 0.75% (95% CI: 0.49 to 1.01). The results from the  Office of National Statistics (ONS) data for infection in England was  0.27% (95% CI: 0.17% to 0.41%). To calculate the risk range of dying following a dental aerosol generating procedure (AGP) is based on the assumption that there is a 100% chance of becoming infected in a single exposure to the virus, the second assumption is that we work with both the minima and the maxima as presented. The summary estimate for asymptomatic individuals in the population was previously calculated as 0.27% (95% CI: 0.12% to 0.45%) (Howe, 2020).

Figure 2. Frequency tree for maximum  and minimum AGP risk

Minimax

With this new range of asymptomatic risk as 1:540 to1:5000 we can calculate the risk that this infected patient passes on the virus to another patient or member of staff and they unfortunately die. We do not treat symptomatic Covid-19 patients in this model. The final range of AGP related death ranges from 1: 54000 to 1:25,000,000 (See Table 3.).

Table 3. Minimum and maximum infection fatality risk

IFR Review paper Minimum and maximum risk of death(natural frequency)
Ioannidis, 2020 1: 135,000 to 1:25,000,000
Meyerowitz-Katz and Merone, 2020 1: 54,000   to 1:1,020,000

Discussion

From the data presented we can make broad assumptions about the population most at risk form corona virus, namely the urban elderly with multiple chronic health conditions. We can also make a similar broad assumption that rural and less densely populated areas have had lower mortality rates. From the current IFR data the variability in trying to make an estimate of risk become incredibly wide from 1:54,000 to 1:25,000,000 which is a 462 times increase in risk. From this result we can see that even in the worst-case scenario the risk of a dental AGP being directly linked to an individual patients death is very low if good cross infection policy is employed which includes PPE as a component of the process. High level PPE (Fluid resistant disposable gowns with plastic aprons, FFP3 and visors) can soon lose its protective benefits with prolonged use, lack of comfort, complicated workflow and difficulties in removing (Phan et al., 2019a, Phan et al., 2019b). There is a huge variance in the volume of aerosol generated in a dental procedure from polishing a composite to removing a broken bridge. It is important in that case that any guidance developed around the use of PPE should consider this variability and set an absolute minimum standard, thus allowing the clinician to tailor the PPE requirement to local conditions/risk. Low risk AGPs could require a fluid resistant face mask and face shield/visor, and as the risk increases higher levels of PPE are added rather than going to maximum PPE in all circumstances (high volume suction is common to all procedures). Wasted resources will harm health care provision to the most vulnerable social groups disproportionally, so high quality studies will need to be rapidly developed to assess these risks.

 

EVERSON, S. A., MATY, S. C., LYNCH, J. W. & KAPLAN, G. A. 2002. Epidemiologic evidence for the relation between socioeconomic status and depression, obesity, and diabetes. Journal of psychosomatic research, 53, 891-895.

HOWE, M. S. 2020. Are we sleepwalking into PPE paralysis? [Online]. Available: https://www.nationalelfservice.net/dentistry/dental-workforce/sleepwalking-ppe-paralysis/ [Accessed].

IOANNIDIS, J. P. 2020. The infection fatality rate of COVID-19 inferred from seroprevalence data.

ISARIC. 2020. Data PlatformCOVID-19 Report 6 MAY 20 [Online]. Available: https://media.tghn.org/medialibrary/2020/05/ISARIC_Data_Platform_COVID-19_Report_6MAY20.pdf [Accessed].

MERLO, J., MULINARI, S., WEMRELL, M., SUBRAMANIAN, S. V. & HEDBLAD, B. 2017. The tyranny of the averages and the indiscriminate use of risk factors in public health: The case of coronary heart disease. SSM Popul Health, 3, 684-698.

MEYEROWITZ-KATZ, G. & MERONE, L. 2020. A systematic review and meta-analysis of published research data on COVID-19 infection-fatality rates. medRxiv.

PHAN, L. T., MAITA, D., MORTIZ, D. C., WEBER, R., FRITZEN-PEDICINI, C., BLEASDALE, S. C., JONES, R. M. & PROGRAM, C. P. E. 2019a. Personal protective equipment doffing practices of healthcare workers. Journal of occupational and environmental hygiene, 16, 575-581.

PHAN, L. T., SWEENEY, D., MAITA, D., MORITZ, D. C., BLEASDALE, S. C., JONES, R. M. & PROGRAM, C. D. C. P. E. 2019b. Respiratory viruses on personal protective equipment and bodies of healthcare workers. Infect Control Hosp Epidemiol, 40, 1356-1360.

POLYA, G. 1945. How to solve it: A new aspect of mathematical method, Princeton university press.

WEBB, H. M., NÁPOLES, A. & PÉREZ-STABLE, E. 2020. COVID-19 and Racial/Ethnic Disparities. JAMA.

Re-opening of dental services: A rapid review of international sources. Part II.

Separating the signal from the noise regarding masks.

Link to the Dental Elf

File:Face Masks used to prevent the spread of Coronavirus in ...

Bottom Line

As the number of clinical guidelines and standard operating procedures increases, we are seeing a reduction in consensus regarding a clear way forward in patient management. If we are going to take an evidence-based approach in a land devoid of direct evidence the policy makers are going to have to defer to the clinical expertise, and local knowledge of the profession in regards to the choice of PPE regarding Covid-19 negative patients requiring aerosol generating procedures.

Background

The recent ‘Cochrane Recommendations for the re-opening of dental services: a rapid review of international sources’ has been updated  on the 16th May to include a further 5 international guidelines. The document now reviews 17 guidance documents from 16 countries (France, Spain, Portugal, Austria, Switzerland, Belgium, Netherlands, Norway, Denmark, Malta, America CDC, America ADA, Canada, Australia, New Zealand, India). The common themes and the relevant recommendations were divided into 5 domains:

  1. Practice preparation and patient considerations.
  2. Personal protective equipment (PPE) for dental practice personnel.
  3. Management of the clinical room.
  4. Dental procedures.
  5. Post-operative cleaning/disinfection/waste management.

Methods

The data was extracted in a similar way to the previous review. The five domains have mostly remained the same with little change in domains 1, 3, 4, and 5. The exception being domain 2 (PPE for dental practice personnel). The domain almost doubled in size to 29 subgroups from the original 15, but at the same time there was a reduction in the consensus with the mean dropping from 58% to 30% (See Figure 1.).

Update_f1

As with the previous review I selected the subgroups that achieved to filter out the large number of subgroups with a low degree of consensus. In the original paper there were 10 subgroups that scored ≥ 50%, in the update this reduced to 7 which constitutes a 42% reduction in agreement considering the expansion of the domain. (See Figure 2).

Figure 2. Subgroups with greater than 50% consensus.

Update_f2

 

Looking at these subgroup headings there is a lot of duplication especially regarding the use of FFP2 and FFP3 masks and the infective status of the patient. To try and clarify this matter a subgroup analysis was undertaken (See Figure 3,).

Figure 3. Mask selection

update_f3

From this chart it was obvious that for Covid negative patients the consensus was for the use of surgical masks, and with Covid infective patients requiring an aerosol generating procedure (AGP) an FFP3 mask should be used. There was however a grey area around the use of a surgical mask combined with a face shield/visor, and an FFP2 mask for Covid negative patients requiring an AGP, the prevalence of asymptomatic patients being very low in the community (ONS, 2020). There was 41% (95% CI: 18% to 65%) agreement regarding the surgical masks, against 59% (95% CI: 35% to 82%) agreement for the use of an FFP2 mask, however as the sample size was small (n=17) the result was not statistically significant (p= 0.29).

Discussion

What can we conclude from the update? Due to the lack of hard evidence regarding the effectiveness of PPE in the real-world clinical environment the guidelines would appear to be based on opinion that ranges from the precautionary principle that all patients should be considered infective, to a more pragmatic approach based on the professions current cross infection strategies. In a perfect world we would like to see well  design randomised controlled studies to answer these questions, but this does not address the here-and-now of real-world dentistry. If we are going to follow the principles of evidence-based dentistry (Sackett et al., 1996) the clinicians are going have to make the decision of using surgical masks, or FFP2 masks  for AGPs based on individual clinical expertise, best available evidence, and their patients values and preferences rather than rigid guidelines that can’t adapt to local circumstances.

References

ONS. 2020. Coronavirus (COVID-19) Infection Survey pilot: England, 14 May 2020 [Online]. Available: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/england14may2020 [Accessed].

SACKETT, D. L., ROSENBERG, W. M., GRAY, J. M., HAYNES, R. B. & RICHARDSON, W. S. 1996. Evidence based medicine: what it is and what it isn’t. British Medical Journal Publishing Group.

https://oralhealth.cochrane.org/news/recommendations-re-opening-dental-services-rapid-review-international-sources

Please may I take your temperature. Screening for Covid-19?

smurf

Bottom line

Even though handheld infrared thermometers are convenient to use to check if a patient has an elevated body temperature, they aren’t sufficiently accurate for screening purposes. Using recent Office of National Statistics data on the prevalence of Covid-19 in the population the false positive rate is too high (>95%). The major confounders regarding accuracy are environmental temperature, humidity, gender, exercise, and age.

Background

In the recent Cochrane ‘Recommendations for the re-opening of dental services: a rapid review of international sources’(Cochrane, 2020) some of the guidelines recommended temperature screening of the patients at reception for elevated body temperature. The rational being that if a patient is infected with Covid -19 the body’s response to the virus often results in core body temperature increase. Traditionally body temperature was taken with a glass/mercury or electronic thermometer that required intimate contact with the patient but now there are handheld infrared thermometers (HIRT) that are quick and require only skin contact via the ear canal, or contactless by measuring the forehead skin temperature. In this opinion paper we aim to find out how effective temperature screening is in detecting Covid infected patients.

Methods

To reduced unnecessary searching through the literature to answer this question the diagnostic accuracy data was extracted from two rapid reviews, the most recent from the Emergency Care Research Institute (ECRI, 2020), and the second from the Canadian Agency for Drugs and Technologies in Health (CADTH, 2014). It was possible to extract the sensitivity and specificity data for 10 studies measuring the effectiveness of HIRT for forehead temperature (FT), and 7 studies measuring ear (tympanic) temperature (TT). The data was extracted and back transformed into a classic 2×2 table giving us the true positive (TP), false negative (FN), false positive (FP) and true negative (TN) data and meta-analysis was carried out using the ‘mada’ package in R. The summary estimate for sensitivity and specificity for TT and FT are tabulated below (See Table 1).

Table 1. Summary estimates for IR thermometer

Measurement location Sensitivity Specificity
Tympanic temperature 78.7 (95% CI: 69.4 to 85.8) 91.8 (95% CI: 75.7 to 97.6)
Forehead temperature 51.1 (95% CI: 19.3 to 82.0) 97.1 (95% CI: 92.2 to 99.0)

The results for the TT and TF results were plotted together on to a Summary Receiver Operating Characteristic (sROC) curve for comparison. The y-axis represents the sensitivity (1.0 =100%), and the x-axis represents 1- specificity (0.1 = 10%), the solid triangle and circle are the summary estimates, and the ellipses are the 95% confidence areas. ( Figure 1.)

Figure 1. Comparison of diagnostic accuracy tests

ROCtemp

A perfect diagnostic test would be in the extreme top left corner representing 100% true positives and 0% false positives and from the chart we can see that point is outside the 95% confidence area meaning that both tests are poor for screening. To clarify this point I transformed the sensitivity/specificity results into a frequency tree (Figure 2) using a diagnostic test calculator (http://araw.mede.uic.edu/cgi-bin/testcalc.pl)

 Figure 2. Frequency trees for diagnostic tests for screening Covid-19 (Prevalence 1:400)

Frequency tree

If these thermometers are used for screening patient for Covid infection then out of every  838 patients who test positive with elevated TT only 20 will be infected which corresponds to a probability of 2.4% and for the FT that rises to 4.2%.

What happens if we use these thermometers to confirm a diagnosis of fever where we set a prevalence of 95% instead of 0.0025% (Figure 3.).

Figure 3. Frequency trees for diagnostic tests to confirm fever

Frequency tree_v1

Conclusions  The authors of the  recent ECRI report concluded:-

Temperature screening programs using IR alone or with a questionnaire for mass screening are ineffective for detecting infected persons, based on our review of evidence from 2 large systematic reviews (SRs), 3 simulation studies, and 6 diagnostic cohort studies (not included in the SRs). Under best-case scenarios, simulation studies suggest such screening will miss more than half of infected individuals. They are ineffective for mass screening because of the low number of infected individuals who have fever at the time of screening and inconsistent technique by operators.

Comments

Both pieces of diagnostic equipment produce highly variable results whether used for screening or confirming a diagnosis of febrile illness. The limitations are well described in the CADTH report (CADTH, 2014):

The retrieved studies have mentioned potential confounders for measure of temperature such as sweat, gender, age, the range of temperature, the rater, physical activity, the use of antipyretic drugs and emotional state. These factors are even more susceptible to vary in a real world conditions than in a clinical study setting. Moreover, the different brand/model/mode of devices used make it difficult to draw general conclusions on a class of thermometers. Also, a fair number of pediatric studies were included in the present review, limiting the extrapolation of their results to a general population.

In conclusion if a patients temperature needs to be taken then tympanic temperature is more reliable than forehead temperature, however its use for screening in the practice creates another layer of complexity in the cross infect/record keeping process with little diagnostic value.

Disclaimer:  The article has not been peer-reviewed; it should not replace individual clinical judgement, and the sources cited should be checked. The views expressed in this commentary represent the views of the author and not necessarily those of the host institution. The views are not a substitute for professional advice.

 

CADTH. 2014. Non-Contact Thermometers for Detecting Fever: A Review of Clinical Effectiveness [Online]. Available: https://www.cadth.ca/infrared-thermometers-detecting-fever-clinical-effectiveness [Accessed].

COCHRANE. 2020. Recommendations for the re-opening of dental services: a rapid review of international sources [Online]. Available: https://oralhealth.cochrane.org/news/recommendations-re-opening-dental-services-rapid-review-international-sources [Accessed].

ECRI. 2020. Infrared Temperature Screening to Identify Potentially Infected Staff or Visitors Presenting to Healthcare Facilities during Infectious Disease Outbreaks [Online]. Available: https://www.ecri.org/covid-webinar-infrared-temperature-screening-reduce-infection-transmission/ [Accessed].