Original Article
A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered

https://doi.org/10.1016/j.jclinepi.2008.10.001Get rights and content

Abstract

Objective

To illustrate the potential and challenges of the simultaneous analysis of a network of trials, using as a case study the investigation of the relative effectiveness of four topical fluoride treatments and two control interventions (placebo and no treatment) in preventing dental caries in children.

Study Design and Setting

We performed multiple-treatments meta-analysis within a Bayesian framework by synthesizing six Cochrane reviews. We explored the compatibility between direct and indirect evidence and adjusted the results using a meta-regression model to take into account differences in the year of randomization across studies.

Results

The validity of our conclusions for the superiority of fluoride toothpaste as indicated from the initial network analysis using Bayesian methods was challenged when we adjusted for possible confounders. The network was dominated by studies comparing placebo with toothpaste, which were older and had been carried out in populations with higher baseline risk than studies involving other fluoride modalities.

Conclusion

After adjusting for possible differences across studies, we did not find clear evidence that any topical fluoride modality is more effective than any other. Multiple-treatments meta-analysis methods allow for more detailed investigations than naïve methods in the analysis of indirect evidence on treatment effects.

Section snippets

Background

Methods for combining clinical trials making different treatment comparisons were first described explicitly over a decade ago [1]. Only recently, however, have they become more widely implemented, with the increased complexity of analyses that underpin clinical guidelines and health technology appraisals, such as those produced by the UK National Institute for Health and Clinical Excellence. We refer to these joint analyses as multiple-treatments meta-analyses. They are also known as ‘mixed

Topical fluoride therapies for preventing dental caries

The use of fluoride has greatly reduced tooth decay in the last few decades [13]. Systemic (ingested) fluoride therapies (e.g., water fluoridation) and topical fluoride therapies (e.g., fluoride toothpaste) are in common use throughout the world, either alone or in combination. The use of topically applied fluoride products, which are much more concentrated than the fluoride in drinking water, has increased over recent decades, and fluoride-containing toothpastes (dentifrices), mouth rinses,

The multiple-treatments meta-analysis model

Consider a study i that compares toothpaste (T) with rinse (R). The estimated treatment effect in this study is the SMD (toothpaste - rinse), denoted by yTR,i, with estimated variance, sTR,i2. The estimates are assumed to be normally distributed around the true SMD, δTR,i:yTR,iN(δTR,i,sTR,i2).Given multiple studies of toothpaste vs. rinse, classical meta-analysis models assume either δTR,i=δTR for a fixed-effect model, or δTR,iN(δTR,τTR2) for a random-effects model. The variance parameter τTR2

Confounding and underlying assumptions of the network analysis

Joint analysis of the data in a multiple-treatments meta-analysis framework allows novel inferences on treatment comparisons that have not been addressed directly in any studies, and it increases precision for comparisons with few data. However, such gains do not come without strong assumptions. The validity of Equation (1) depends critically on there being no substantive differences between the sources of evidence that inform δTR, δTP, and δRP. Consider a multiple-treatments meta-analysis of

Estimation of incoherence

We now discuss some ‘signals’ of the violation of the network assumptions and some ways to address them. We use the term coherence to describe the presence of agreement between direct and indirect evidence, and incoherence for the converse. For example, suppose we have evidence on the direct comparison δTRD from a meta-analysis of all trials of head-to-head comparisons of toothpaste and rinse and evidence on the indirect comparison δTRI=δTPDδRPD from the difference between meta-analyses of

Implementation

We implement the network model using Bayesian methods in WinBUGS [27], mainly because of the natural way in which full uncertainty in all model parameters can be accounted for. A particular advantage of using a Bayesian framework, however, is the straightforward ability to rank the treatments by calculating the probability that each intervention has the largest treatment effect. Because our use of a Bayesian framework is for convenience rather than because we wish to incorporate prior

Pairwise comparison and multiple-treatments meta-analysis for the fluoride data

We first analysed each comparison separately. There are 150 possible comparisons: 121 from the two arm trials, 24 from the eight three-arm trials, and five from the four-armed trials (toothpaste and placebo are not to be compared with no treatment). The SMDs and median heterogeneity standard deviations (specific to each comparison) are given in Table 2.

We then analysed jointly all 140 independent comparisons from the 130 trials in a multiple-treatments meta-analysis: 121 from the two arm

Conclusions and limitations

We have analyzed simultaneously a complex network of clinical trials involving four fluoride modalities and two control interventions with the aim of determining the most effective intervention. The validity of our conclusions from an initial network analysis, indicating superiority of toothpaste, was challenged when we adjusted for possible confounders. Studies supporting the effectiveness of toothpaste were older, and have been carried out in populations with higher baseline risk compared

References (27)

  • M. Kyrgiou et al.

    Survival benefits with diverse chemotherapy regimens for ovarian cancer: meta-analysis of multiple treatments

    J Natl Cancer Inst

    (2006)
  • S. Heres et al.

    Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics

    Am J Psychiatry

    (2006)
  • B.M. Psaty et al.

    Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis

    JAMA

    (2003)
  • Cited by (236)

    View all citing articles on Scopus
    View full text