Maxillary sinus augmentation – relative performance of available biomaterials and the challenge of small studies

To allow dental implant placement in the posterior maxilla, it is sometimes necessary to increase the height of the residual alveolar bone of the maxillary sinus floor by undertaking a sinus lift procedure. The historical material of choice has been autogenous bone (AB), but this can lead to donor-site morbidity following harvesting. To simplify this process of sinus augmentation, several substitute materials have been studied, such as xenografts in the form of deproteinised bovine bone, synthetic grafts, growth factors, and platelet concentrates. There have been three recent systematic reviews utilising standard pairwise meta-analyses to investigate the efficacy of these biomaterials as a substitute for AB (Corbella et al., 2016; Danesh-Sani et al., 2017; Ting et al., 2017). The authors chose to undertake a Bayesian network meta-analysis (NMA) to evaluate and rank all these materials simultaneously in their capacity to form new bone. (Trimmel et al., 2021)

Methods

The study protocol was registered in PROSPERO (International Prospective Register of Systematic Reviews) and followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis for Network Meta-analysis (PRISMA-NMA) guidelines (Hutton et al., 2015). A systematic search for suitable randomised control trials (RCTs) was performed in the  Cochrane Library (CENTRAL), EBSCO, Embase, MEDLINE (via PubMed), and Web of Science Core Collection electronic databases with records published up to October 1, 2019. The risk of bias was assessed using the Cochrane Risk of Bias Tool.

The Bayesian approach for NMAs describes the range and probability of the parameter of interest (e.g., treatment effect here being bone % bone regeneration). The posterior distribution produced by this method predicts the new range and probability of plausible values for these parameters with the representation of uncertainty in the form of a 95% credibility interval. The interventions were ranked by their posterior probability by calculating the surface under the cumulative ranking (SUCRA) curve values.

Results

• 34 RCTs (842 maxillary sinus augmentations) with an average healing period of 5–8 months were included in the NMA. 31 were two-arm studies, and 3 were three-arm studies.
• There were 28 treatment options, 378 possible pairwise comparisons, and 31 pairwise comparisons using direct data.
• The overall assessment for risk of bias showed low risk in 5 studies, unclear risk in 20 studies, and high risk in 9 studies.
• There were significant differences favouring the bovine bone + bone marrow concentrate (BMC) composite graft and the biodegradable copolymer; and between the bovine + BMC composite graft and the allograft.
• From the 376 pairwise comparisons, no significant differences were detected, leading to a rejection of the null hypothesis that AB alone is the most favourable material for bone regeneration.
• The SUCRA ranking probability for the most effective bone grafting material for new bone regeneration:-

Top Five Grafting Materials	SUCRA ranking
Bovine xenograft + bone marrow concentrate (BMC)	81%
Bovine xenograft + platelet-rich plasma (PRP)	77%
Bioactive glass ceramic + autologous bone 1:1	70%
Nanocrystalline hydroxyapatite in silica gel	70%
Bioactive glass-ceramic	70%
Autologous bone graft	57%

Conclusions

The authors concluded:-

The results of the present NMA suggest that the use of biomaterials does not result in a statistically significant difference in the rate of NB formation compared to AB alone as grafting material. However, their use can significantly reduce the amount of AB graft required for MSA, resulting in a less invasive surgical intervention and shorter surgical time. The combination of biomaterials with AB or autologous cell concentrates, such as BMC, PRP, and platelet-rich fibrin, represents a feasible alternative for AB substitution to achieve high NB formation. The superiority of AB compared to biomaterials for MSA in a healing time frame of 5–8 months cannot be justified.

Comments

Network meta-analyses are highly complex statistical tools to evaluate multiple treatment options. This complexity can limit the strength and certainty of the inferences produced even when the NMA is well done, as in this case. In a two-part paper by Foote, Chaudhry and co-workers, they outline a practical guide on interpreting NMAs (Foote et al., 2015; Chaudhry et al., 2015). It should be noted that even though this NMA had a high number of treatment nodes, it was a sparse network with a network density of about 10%, whereas a full connected (dense) network would achieve 100%. The sparsity of connections increases the reliance on indirect evidence and the effects of heterogeneity within the included studies, leading to extremely wide confidence/credibility intervals and questionable results (Brignardello-Petersen et al., 2019). To explore this potential problem, the primary data in Table 2 of Trimmels paper was extracted and reanalysed using the R package called “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct a Network meta-analysis) in R (Béliveau et al., 2019). The initial reanalysis duplicated Trimmels results, as can be seen in the netplot (Figure 1).

Figure 1. Netplot of Trimmel original data.

The material and methods section of the original paper mentioned that each intervention would be presented compared to a placebo in a forest plot; however, the forest plot was not shown in the paper. In the reanalysis, the forest plot is given below. It clearly shows the very wide credibility intervals. Almost all the treatment options cross the null effect line, confirming the problems created by incorporating many small poorly networked studies and the resultant indirect estimates the model generates (Figure 2.).

Figure 2. Forest plot relative to autogenous bone

To explore this further, the simplest method was to undertake a sensitivity meta-analysis and remove those studies considered to be at high risk of bias. These studies would be the most likely to result in misleading results (Chaudhry et al., 2015). The 9 papers the authors considered to be a high risk of bias were removed from the NMA database, and 2 further papers that appeared to share a control group. The rankings were then recalculated and presented alongside the original ranking data to observe any changes. The reanalysis removed 3 treatments from the ranking (autogenous bone plus autologous platelet concentrate, bovine plus bone marrow aspirates, and porcine bone), plus bone marrow concentrate was dropped 18 places from 81% to 40%. The top five highest-ranking treatments now include bovine bone mixed with autologous bone, bovine bone plus platelet-rich fibrin (PRF), and biphasic calcium phosphate (HA/β-TCP = 60/40) combined with fibrin sealant (FS). (Table 1.)

Table 1. Change in top 5 ranked augmentation materials.

Rank	Sensitivity meta-analysis	Score	Original meta-analysis	Score
1	Bovine+AB4:1	85	Bovine+BMC	81
2	Bovine+AB1:1	83	Bovine+PRP	77
3	Bovine+AB1:1+laser stimulation	77	Bioglass+AB1:1	70
4	Bovine+PRP	76	HA+silicalgel	70
5	BCP+FS	73	Bioglass	70

We could conclude that the sensitivity analysis confirmed the authors finding that autologous bone did not show superiority to composite grafting material. Significantly, however, the ranking of those materials changes at the extremes, with the first six highest rankings being substantially downgraded and three treatments being removed from the meta-analysis altogether (Figure 3).

Figure 3. Change in SUCRA scores with sensitivity analysis.

In summary, both the researcher and the reader must exercise caution when undertaking a network meta-analysis. Leaving aside the issue of transitivity assumptions, consistency and statistical complexity,  network analysis will not eliminate the problems associated with combining multiple small, severely underpowered studies that could be potentially at high risk of bias. To quote Foote and co-authors: –

Assessing the credibility of the methodology is an important first step in critically appraising an NMA. As with conventional systematic reviews, assessing credibility involves evaluating the article for a sensible research question, an exhaustive search, reproducible selection and assessment of articles, presenting clinically relevant results, and addressing certainty in effect estimates (Foote et al., 2015).

Links

Primary paper

Trimmel, B., Gede, N., Hegyi, P., et al. 2021. Relative Performance of Various Biomaterials Used for Maxillary Sinus Augmentation: A Bayesian Network Meta‐Analysis. Clinical Oral Implants Research, 32, 135-153.

Review protocol in PROSPERO

Other references

Béliveau, A., Boyne, D. J., Slater, J., et al. 2019. Bugsnet: An R Package to Facilitate the Conduct and Reporting of Bayesian Network Meta-Analyses. BMC Medical Research Methodology, 19.

Brignardello-Petersen, R., Murad, M. H., Walter, S. D., et al. 2019. Grade Approach to Rate the Certainty from a Network Meta-Analysis: Avoiding Spurious Judgments of Imprecision in Sparse Networks. J Clin Epidemiol, 105, 60-67.

Chaudhry, H., Foote, C. J., Guyatt, G., et al. 2015. Network Meta-Analysis: Users’ Guide for Surgeons: Part Ii – Certainty. Clinical Orthopaedics & Related Research, 473, 2172-2178.

Corbella, S., Taschieri, S., Weinstein, R., et al. 2016. Histomorphometric Outcomes after Lateral Sinus Floor Elevation Procedure: A Systematic Review of the Literature and Meta-Analysis. Clinical Oral Implants Research, 27, 1106-1122.

Danesh-Sani, S. A., Engebretson, S. P. & Janal, M. N. 2017. Histomorphometric Results of Different Grafting Materials and Effect of Healing Time on Bone Maturation after Sinus Floor Augmentation: A Systematic Review and Meta-Analysis. Journal of Periodontal Research, 52, 301-312.

Foote, C. J., Chaudhry, H., Bhandari, M., et al. 2015. Network Meta-Analysis: Users’ Guide for Surgeons: Part I – Credibility. Clinical Orthopaedics & Related Research, 473, 2166-2171.

Hutton, B., Salanti, G., Caldwell, D. M., et al. 2015. The Prisma Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-Analyses of Health Care Interventions: Checklist and Explanations. Ann Intern Med, 162, 777-784.

Ting, M., Rice, J. G., Braid, S. M., et al. 2017. Maxillary Sinus Augmentation for Dental Implant Rehabilitation of the Edentulous Ridge: A Comprehensive Overview of Systematic Reviews. Implant Dent, 26, 438-464.