Clinicians rely heavily on consensus guidelines when they develop treatment protocols for their patients. The idea is that the most experienced clinicians and academics meet and systematically develop what they feel is best practice in their particular field. Health care providers and regulators then rely on this distillation of knowledge, experience and evidence in their decision making. A systematic review of guideline quality in peer-reviewed medical literature was conducted in 1999 and found only a 43% adherence to reporting standards (Shaneyfelt et al. 1999). In 2003, a generic tool was created by the AGREE (Appraisal of Guidelines, REsearch and Evaluation) collaboration to create “common standards to improve the quality process and reporting of guideline development” (Cluzeau et al. 2003). With use the protocol evolved into AGREE II (Brouwers et al. 2010) which is made up of 23 items in 6 domains:
- Scope and Purpose
- Stakeholder Involvement
- Rigour of Development
- Clarity of Presentation
- Applicability
- Editorial Independence
The research question reference consensus guidelines in implant dentistry was: “Do consensus guidelines published in high ranked implant journals meet the requirements proposed in the AGREE II instrument? A secondary objective was to evaluate whether the inclusion of systematic reviews conducted to support the consensus guidelines improved their methodological quality.
Methods
Two authors (KA,MA) independently searched and evaluated consensus guidelines from the 6 highest impact implant dentistry journals (assigned by Journal Citation Reports) and Medline database via PubMed. The search was limited to May 2009 – February 2016 in line with the publication of AGREE II. Reasons for papers to be excluded was recorded.
Four authors (KA,TA,LM,MA), following assessor training then independently applied the AGREE II tools to the consensus guidelines as described in the user’s manual. Domain scores were presented as median percentages of the maximum possible with their respective interquartile ranges. Domain scores were divided into consensus guidelines, and consensus guidelines with systematic reviews.
Results
- From an initial 258 publications 27 consensus guidelines fulfilled the inclusion criteria of which 19 was included for comparison
- The journals were: Clinical Oral Implants Research (COIR), Clinical Implant Dentistry and Related Research (CIDRR), European Journal of Oral Implants (EJOI), The International Journal of Oral and Maxillofacial Implants (JOMI), Journal of Oral Implantology, and Implant Dentistry.
- 26 guidelines were developed after meetings in Europe.
- The European Association of Osseointigration (EAO) developed the most guidelines (n=9)
- The number of authors ranged from 2-27 (median, 9)
Primary Outcomes
| AGREE II Domains | Consensus guidelines only.
Median is % of maximum score |
Consensus guidelines plus systematic reviews.
|
| Domain 4 (Clarity of presentation) | Median, 75; (IQR 15.30) | Median, 84.70;(IQR, 9.80). |
| Domain 1 (Scope and Purpose) | Median, 69.40; (IQR, 36.20). | Median, 79.20; (IQR, 73) |
| Domain 2 (Stakeholder Involvement) | Median, 41.70; (IQR, 17.70) | Median, 76.40; (IQR,18.10) |
| Domain 6 (Editorial Independence) | Median, 41.7 ;(IQR, 83.30) | Median, 56.30 (IQR, 34.40) |
| Domain 3 (Rigour of Development) | Median, 30.70; (IQR, 26.50) | Median, 50 (IQR,44.40) |
| Domain 5 (Applicability) | Median, 26;(IQR, 12.50) | Median, 26; (IQR, 20.80) |
Conclusion (the author concluded)
“Methodological improvement of consensus guidelines published in major implant dentistry journals is needed. The findings of the present study may help researchers to better develop consensus guidelines in implant dentistry, which will improve the quality and trust of information needed to make proper clinical decisions”.
Comments
For the clinician in practice placing or restoring on dental implants this is a very important paper and goes a long way in explaining the disjoint between what is presented on the international conference circuit and what we see in day-day practice.
To help with interpreting the authors data I have modified it slightly to present the data as median with 95% confidence intervals and place the meta-analysis it in a forest plot.
The first chart show the AGREE II scores relating to the consensus guidelines that did not include the use of systematic reviews. The highest scoring domains were Domain 4. (Clarity of Presentation) and Domain 1. (Scope and Purpose) which score in the 70’s. There is a gap and the next four Domains in rank order are; 6. (Editorial Independence), 2. (Stakeholder Involvement), 3. (Rigour of Development) and lastly 6. (Applicability). These narrative guidelines therefore only fulfil 50% of the AGREE II criteria.
The second chart show a meta-analysis comparing the narrative guideline scores compared with the consensus guidelines + systematic reviews, and there is roughly a 9% improvement bringing the score up to 60% compliance with AGREE II.
What becomes obvious however is that 18% improvement came out of Domain 3. (Rigour of Development) but 34% from Domain 6. (Editorial Independence). Going back then to the meta-analysis of consensus guidelines + systematic reviews the improvement has been weak and polarises the results.
From a clinicians point of view what options do we have available to us to improve consensus guidelines?
- Domains 1 (Scope and Purpose) and 4 (Clarity of Presentation) are good, we know what questions to ask and how to present them.
- Domain 6 (Editorial Independence) is improving but there still needs to be more clarity over the editorial independence from funding bodies and conflicts of interest from the funding bodies and authors.
- Domain 3. (Rigour of Development) More effort needs to be spent in this area as is is the foundation on which the guidelines are built, based on validated systematic methods for assessing and analysing the relevant primary research. Analysis also needs to include health benefits, side effects and complications. The guidelines should also be externally validated prior to publication.
- Domain 2 (Stakeholder Involvement) at present is dominated by University professors and private practitioners with strong academic connections. This area would benefit from the views of clinicians working outside of specialist practice, health care providers, funders and patient groups.
- Domain 6. (Applicability) is the weakest domain, this may be in part due to the lack of input at the stakeholder level . For clinicians in practice, and their patients cost implications are a major barrier to applying guideline recommendations.
Links
Primary paper
Faggion, C.M.J. et al., 2017. Methodological Quality of Consensus Guidelines in Implant Dentistry. PloS one, 12(1), p.e0170262. Available at: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=prem&NEWS=N&AN=28107405.
Other references
Brouwers, M.C. et al., 2010. AGREE II: Advancing guideline development, reporting and evaluation in health care. Journal of Clinical Epidemiology, 63(12), pp.1308–1311.
Cluzeau, F. et al., 2003. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Quality & safety in health care, 12(1), pp.18–23. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1743672&tool=pmcentrez&rendertype=abstract.
Shaneyfelt, T.M., Mayo-Smith, M.F. & Rothwangl, J., 1999. Are guidelines following guidelines? JAMA: The Journal of the American Medical Association, 281(20), pp.1900–1905.
Thoughts on Life and Dentistry 