Maxillary sinus augmentation – relative performance of available biomaterials and the challenge of small studies

To allow dental implant placement in the posterior maxilla, it is sometimes necessary to increase the height of the residual alveolar bone of the maxillary sinus floor by undertaking a sinus lift procedure. The historical material of choice has been autogenous bone (AB), but this can lead to donor-site morbidity following harvesting. To simplify this process of sinus augmentation, several substitute materials have been studied, such as xenografts in the form of deproteinised bovine bone, synthetic grafts, growth factors, and platelet concentrates. There have been three recent systematic reviews utilising standard pairwise meta-analyses to investigate the efficacy of these biomaterials as a substitute for AB (Corbella et al., 2016; Danesh-Sani et al., 2017; Ting et al., 2017). The authors chose to undertake a Bayesian network meta-analysis (NMA) to evaluate and rank all these materials simultaneously in their capacity to form new bone. (Trimmel et al., 2021)


The study protocol was registered in PROSPERO (International Prospective Register of Systematic Reviews) and followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis for Network Meta-analysis (PRISMA-NMA) guidelines (Hutton et al., 2015). A systematic search for suitable randomised control trials (RCTs) was performed in the  Cochrane Library (CENTRAL), EBSCO, Embase, MEDLINE (via PubMed), and Web of Science Core Collection electronic databases with records published up to October 1, 2019. The risk of bias was assessed using the Cochrane Risk of Bias Tool.

The Bayesian approach for NMAs describes the range and probability of the parameter of interest (e.g., treatment effect here being bone % bone regeneration). The posterior distribution produced by this method predicts the new range and probability of plausible values for these parameters with the representation of uncertainty in the form of a 95% credibility interval. The interventions were ranked by their posterior probability by calculating the surface under the cumulative ranking (SUCRA) curve values.


  • 34 RCTs (842 maxillary sinus augmentations) with an average healing period of 5–8 months were included in the NMA. 31 were two-arm studies, and 3 were three-arm studies.
  • There were 28 treatment options, 378 possible pairwise comparisons, and 31 pairwise comparisons using direct data.
  • The overall assessment for risk of bias showed low risk in 5 studies, unclear risk in 20 studies, and high risk in 9 studies.
  • There were significant differences favouring the bovine bone + bone marrow concentrate (BMC) composite graft and the biodegradable copolymer; and between the bovine + BMC composite graft and the allograft.
  • From the 376 pairwise comparisons, no significant differences were detected, leading to a rejection of the null hypothesis that AB alone is the most favourable material for bone regeneration.
  • The SUCRA ranking probability for the most effective bone grafting material for new bone regeneration:-
Top Five Grafting MaterialsSUCRA ranking
Bovine xenograft + bone marrow concentrate (BMC)81%
Bovine xenograft + platelet-rich plasma (PRP)77%
Bioactive glass ceramic + autologous bone 1:170%
Nanocrystalline hydroxyapatite in silica gel70%
Bioactive glass-ceramic70%
Autologous bone graft57%


The authors concluded:-

The results of the present NMA suggest that the use of biomaterials does not result in a statistically significant difference in the rate of NB formation compared to AB alone as grafting material. However, their use can significantly reduce the amount of AB graft required for MSA, resulting in a less invasive surgical intervention and shorter surgical time. The combination of biomaterials with AB or autologous cell concentrates, such as BMC, PRP, and platelet-rich fibrin, represents a feasible alternative for AB substitution to achieve high NB formation. The superiority of AB compared to biomaterials for MSA in a healing time frame of 5–8 months cannot be justified.


Network meta-analyses are highly complex statistical tools to evaluate multiple treatment options. This complexity can limit the strength and certainty of the inferences produced even when the NMA is well done, as in this case. In a two-part paper by Foote, Chaudhry and co-workers, they outline a practical guide on interpreting NMAs (Foote et al., 2015; Chaudhry et al., 2015). It should be noted that even though this NMA had a high number of treatment nodes, it was a sparse network with a network density of about 10%, whereas a full connected (dense) network would achieve 100%. The sparsity of connections increases the reliance on indirect evidence and the effects of heterogeneity within the included studies, leading to extremely wide confidence/credibility intervals and questionable results (Brignardello-Petersen et al., 2019). To explore this potential problem, the primary data in Table 2 of Trimmels paper was extracted and reanalysed using the R package called “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct a Network meta-analysis) in R (Béliveau et al., 2019). The initial reanalysis duplicated Trimmels results, as can be seen in the netplot (Figure 1).

Figure 1. Netplot of Trimmel original data.

The material and methods section of the original paper mentioned that each intervention would be presented compared to a placebo in a forest plot; however, the forest plot was not shown in the paper. In the reanalysis, the forest plot is given below. It clearly shows the very wide credibility intervals. Almost all the treatment options cross the null effect line, confirming the problems created by incorporating many small poorly networked studies and the resultant indirect estimates the model generates (Figure 2.).

Figure 2. Forest plot relative to autogenous bone

To explore this further, the simplest method was to undertake a sensitivity meta-analysis and remove those studies considered to be at high risk of bias. These studies would be the most likely to result in misleading results (Chaudhry et al., 2015). The 9 papers the authors considered to be a high risk of bias were removed from the NMA database, and 2 further papers that appeared to share a control group. The rankings were then recalculated and presented alongside the original ranking data to observe any changes. The reanalysis removed 3 treatments from the ranking (autogenous bone plus autologous platelet concentrate, bovine plus bone marrow aspirates, and porcine bone), plus bone marrow concentrate was dropped 18 places from 81% to 40%. The top five highest-ranking treatments now include bovine bone mixed with autologous bone, bovine bone plus platelet-rich fibrin (PRF), and biphasic calcium phosphate (HA/β-TCP = 60/40) combined with fibrin sealant (FS). (Table 1.)

Table 1. Change in top 5 ranked augmentation materials.

RankSensitivity meta-analysisScoreOriginal meta-analysisScore
3Bovine+AB1:1+laser stimulation77Bioglass+AB1:170

We could conclude that the sensitivity analysis confirmed the authors finding that autologous bone did not show superiority to composite grafting material. Significantly, however, the ranking of those materials changes at the extremes, with the first six highest rankings being substantially downgraded and three treatments being removed from the meta-analysis altogether (Figure 3).

Figure 3. Change in SUCRA scores with sensitivity analysis.

In summary, both the researcher and the reader must exercise caution when undertaking a network meta-analysis. Leaving aside the issue of transitivity assumptions, consistency and statistical complexity,  network analysis will not eliminate the problems associated with combining multiple small, severely underpowered studies that could be potentially at high risk of bias. To quote Foote and co-authors: –

Assessing the credibility of the methodology is an important first step in critically appraising an NMA. As with conventional systematic reviews, assessing credibility involves evaluating the article for a sensible research question, an exhaustive search, reproducible selection and assessment of articles, presenting clinically relevant results, and addressing certainty in effect estimates (Foote et al., 2015).

Primary paper

Trimmel, B., Gede, N., Hegyi, P., et al. 2021. Relative Performance of Various Biomaterials Used for Maxillary Sinus Augmentation: A Bayesian Network Meta‐Analysis. Clinical Oral Implants Research, 32, 135-153.

Review protocol in PROSPERO

Other references

Béliveau, A., Boyne, D. J., Slater, J., et al. 2019. Bugsnet: An R Package to Facilitate the Conduct and Reporting of Bayesian Network Meta-Analyses. BMC Medical Research Methodology, 19.

Brignardello-Petersen, R., Murad, M. H., Walter, S. D., et al. 2019. Grade Approach to Rate the Certainty from a Network Meta-Analysis: Avoiding Spurious Judgments of Imprecision in Sparse Networks. J Clin Epidemiol, 105, 60-67.

Chaudhry, H., Foote, C. J., Guyatt, G., et al. 2015. Network Meta-Analysis: Users’ Guide for Surgeons: Part Ii – Certainty. Clinical Orthopaedics & Related Research, 473, 2172-2178.

Corbella, S., Taschieri, S., Weinstein, R., et al. 2016. Histomorphometric Outcomes after Lateral Sinus Floor Elevation Procedure: A Systematic Review of the Literature and Meta-Analysis. Clinical Oral Implants Research, 27, 1106-1122.

Danesh-Sani, S. A., Engebretson, S. P. & Janal, M. N. 2017. Histomorphometric Results of Different Grafting Materials and Effect of Healing Time on Bone Maturation after Sinus Floor Augmentation: A Systematic Review and Meta-Analysis. Journal of Periodontal Research, 52, 301-312.

Foote, C. J., Chaudhry, H., Bhandari, M., et al. 2015. Network Meta-Analysis: Users’ Guide for Surgeons: Part I – Credibility. Clinical Orthopaedics & Related Research, 473, 2166-2171.

Hutton, B., Salanti, G., Caldwell, D. M., et al. 2015. The Prisma Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-Analyses of Health Care Interventions: Checklist and Explanations. Ann Intern Med, 162, 777-784.

Ting, M., Rice, J. G., Braid, S. M., et al. 2017. Maxillary Sinus Augmentation for Dental Implant Rehabilitation of the Edentulous Ridge: A Comprehensive Overview of Systematic Reviews. Implant Dent, 26, 438-464.

Endodontic diagnosis CBCT vs. conventional radiography

The diagnosis and treatment of apical periodontitis can be complicated, especially if it is asymptomatic or has complex radicular architecture, and not all apical radiolucencies are inflammatory lesions. Conventional two-dimensional (2D) radiography is the most common imaging technique to identify these lesions, but it cannot always detect every apical lesion due to its size or the presence of other anatomical structures. Cone-beam computed tomography (CBCT)  adds a third dimension to the image (3D), allowing the clinician to visualise the true extent of a periapical lesion in the horizontal plane.  However, this improved diagnostic yield comes at both a cost in additional radiation exposure and overall cost to the patient.

The aim of this study was to compare the diagnostic performance outcomes between conventional 2D radiographs and CBCT in detecting persistent apical disease after root canal treatment. (Ramis-Alario et al., 2021)


The study followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis for Diagnostic Test Accuracy (PRISMA-DTA) guidelines. Two independent authors conducted an electronic search of the Medline via PubMed, Embase, Web of Science and Google Scholar databases for published articles up to December 2019 without language restriction. Opengrey was also searched for non-peer reviewed literature. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) guidelines.


  • Of the 27 papers, 13 articles compared digital periapical radiography (DPR) vs CBCT, two articles compared conventional periapical radiography (CPR) and DPR vs CBCT, two compared DPR and panoramic radiography vs CBCT, five compared CPR vs CBCT, one compared CPR and panoramic radiography vs CBCT, one compared periapical radiography vs CBCT but failed to mention the features of both CBCT and periapical x-ray devices, and three articles compared panoramic radiography vs CBCT.
  • The risk of bias was high in terms of flow and timing but low in terms of applicability.
  • Primary diagnostic measures
    • Sensitivity and specificity for 2-D radiograpgy compared to CBCT. The pooled sensitivity and specificity estimates were 58% (0.58; 95% confidence interval (CI), 0.53 to 0.64) and 100% (1.00; 95% CI, 1.00 to 1.00), respectively.
    • The overall heterogeneity was high for sensitivity (I2 = 97.3%).
    • The pooled area under the receiver operating characteristic curve (AUCROC) discriminating capacity of conventional radiographs to diagnose persistent apical disease was good at 77% (0.77; 95% CI, 0.75 to 0.80).
  • Secondary diagnostic measures
    • Positive and negative predictive values. The pooled positive predictive value estimate was 100% (1.00; 95% CI, 1.00 to 1.00; data not shown). The negative predictive value was 68% (0.68; 95% CI, 0.63 to 0.72)


The authors concluded:-

Moderate certainty evidence suggested that conventional radiographs showed poor sensitivity and excellent specificity but good diagnostic performance in terms of AUCROC and accuracy. Sensitivity, AUCROC, and negative likelihood ratio values could be reduced if the time elapsed to diagnosis after root canal treatment exceeded 5-years. The use of CBCT with a reduced field of view or a 2D radiographic technique should be weighed considering patient-specific and indication-oriented criteria as taking precedence over the therapeutic goal.


This review followed the standard PRISMA-DTA protocol but lacked pre-registration; however, it had a large sample size for meta-analysis and opens up some interesting areas for discussion regarding the cost in terms of harms and benefits of achieving increased diagnostic yield in imaging. To refine the comparison of 3D with 2D radiography, I will limit the discussion to CBCT compared with digital periapical radiography. To add some context, there was a recent paper published comparing CBCT with intraoral periapical lesions in endodontics which concluded that: –

 CBCT imaging was reported in this investigation to have twice the odds of detecting a periapical lesion than traditional periapical radiography in endodontic outcome studies.(Aminoshariae et al., 2018)

It can sometimes be quite challenging to interpret odds ratios as they are not the only way to present an association when the primary outcome is binary. The reader should understand odds ratios in the context of other information, such as the underlying probability (Norton et al., 2018). I have taken the primary data from Aminoshariae and redone the meta-analysis to show the risk difference rather than the odds ratio (Figure 1).

Figure 1.

From these new results, we can see that there is a small but significant increase in the ability of the CBCT to detect apical pathology of 7%  (95%CI: 2% to 11%: Prediction interval -1% to 14%), which appears less dramatic than the also correct doubling of odds (2.04 (95%CI: 1.52 to 2.73). Since the improvement was significant, CBCT was selected as the reference standard for the Ramis-Alarios paper. As the pooled analysis contained a mix of conventional, digital intraoral, and panoramic radiographs, I have calculated a new Summary Reciever Operating Characteristic (SROC) curve using the ‘mada’ package in R. Only the primary data regarding digital periapical radiography data was used. There were zero false positives in the data, so a continuity correction of 1 had to be applied to plot the new SROC curve, which sightly reduced the specificity from 100% to 96%. The new summary estimate for digital periapical radiography produced a sensitivity of 58.5% (95%CI: 48% to 68%) and a specificity of 96% (95%CI: 98% to 93%), the AUCROC  was excellent at 0.90 (Mandrekar, 2010) (Figure 2.).

Figure 2.

The subgroup analysis concluded that digital periapical radiography displayed an excellent diagnostic capacity when compared to CBCT. Therefore the small increase in diagnostic accuracy offered by the CBCT needs to be weighed against the increased effective dose of radiation the patient will be exposed to yield that additional information. To clarify this point a single digital periapical radiograph has an effective dose of 0.005mSv ( 1 day of background ). A current CBCT set at a high resolution for endodontics will have an effective dose ranging from 0.041mSv to 0.143mSv ( 8 to 29 days of background radiation)(Qiang et al., 2019; White et al., 2014). To quote Ramis-Alario:

“The decision to use CBCT or a 2D technique in patients in whom a functional tooth exhibits a periapical radiolucency after conventional root canal treatment should be clearly justified by the clinician to avoid unnecessary radiation exposure to the patient, in keeping with the “as low as diagnostically acceptable” (ALADA) criterion.”(Ramis-Alario et al., 2021)

Primary reference

Ramis-Alario A, Soto-Peñaloza D, Tarazona-Alvarez B, Peñarrocha-Diago M, Peñarrocha-Oltra D. Comparison of the diagnostic efficacy of 2D radiography and cone beam computed tomography in persistent apical periodontal disease: A PRISMA-DTA systematic review and meta-analysis. Oral Surg Oral Med Oral Pathol Oral Radiol. 2021 Oct;132(4):e153-e168. doi: 10.1016/j.oooo.2021.07.002. Epub 2021 Jul 15. PMID: 34376356.

Other references

Aminoshariae A, Kulild JC, Syed A. Cone-beam Computed Tomography Compared with Intraoral Radiographic Lesions in Endodontic Outcome Studies: A Systematic Review. J Endod. 2018 Nov;44(11):1626-1631. doi: 10.1016/j.joen.2018.08.006. PMID: 30409446.

Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010 Sep;5(9):1315-6. doi: 10.1097/JTO.0b013e3181ec173d. PMID: 20736804.

Norton EC, Dowd BE, Maciejewski ML. Odds Ratios-Current Best Practice and Use. JAMA. 2018 Jul 3;320(1):84-85. doi: 10.1001/jama.2018.6971. PMID: 29971384.

Qiang W, Qiang F, Lin L. Estimation of Effective Dose of Dental X-Ray Devices. Radiat Prot Dosimetry. 2019 Jun 1;183(4):417-421. doi: 10.1093/rpd/ncy159. PMID: 30169836.

White SC, Scarfe WC, Schulze RK, Lurie AG, Douglass JM, Farman AG, Law CS, Levin MD, Sauer RA, Valachovic RW, Zeller GG, Goske MJ. The Image Gently in Dentistry campaign: promotion of responsible use of maxillofacial radiology in dentistry for children. Oral Surg Oral Med Oral Pathol Oral Radiol. 2014 Sep;118(3):257-61. doi: 10.1016/j.oooo.2014.06.001. Epub 2014 Jun 16. PMID: 25066244.