Skip to main content

Biostatistics in Practice: Principles and Procedures - A MedCalc companion e-book

Meta-analysis

Clinical research is cumulative. Individual studies address the same question independently, sometimes reaching different conclusions because of small sample sizes, different patient populations, or chance variation. A meta-analysis resolves this by systematically combining the quantitative findings from separate but comparable studies into a single, more precise numerical estimate of the overall effect of interest.

The logic is straightforward: if ten trials each examine the same drug but each enrols only 50 patients, their individual estimates will be imprecise and may conflict. Pooling all 500 patients' worth of information — carefully weighted to account for each study's precision — yields a much narrower confidence interval and a more reliable answer. Meta-analysis is therefore not merely a convenience; it is often the highest level of evidence available on a clinical question.

Meta-analysis can be applied to many types of outcome, including means of continuous variables, correlations, odds ratios, relative risks, risk differences, proportions, and areas under ROC curves.

Weighting

Not all studies contribute equally to the pooled estimate. A large, carefully conducted trial with 2,000 participants deserves more influence than a small pilot study of 30. Meta-analysis formalises this intuition through inverse-variance weighting: each study receives a weight equal to the reciprocal of its squared standard error (i.e. the reciprocal of its variance):

$$ w_i = \frac{1}{SE_i^2} = \frac{1}{v_i} $$

where $SE_i$ is the standard error of the effect estimate from study $i$ and $v_i = SE_i^2$ is its variance. Studies with smaller standard errors (larger samples, more events) receive larger weights and therefore exert greater influence on the pooled result.

The weighted average of the $k$ effect estimates $\theta_1, \theta_2, \ldots, \theta_k$ is:

$$ \bar{\theta}_w = \frac{\displaystyle\sum_{i=1}^{k} w_i\,\theta_i}{\displaystyle\sum_{i=1}^{k} w_i} $$

The standard error of the pooled estimate — and hence its confidence interval — follows directly:

$$ SE(\bar{\theta}_w) = \frac{1}{\sqrt{\displaystyle\sum_{i=1}^{k} w_i}} \qquad \text{CI} = \bar{\theta}_w \pm z^* \cdot SE(\bar{\theta}_w) $$

In the forest plot below, the area of each square marker is proportional to the weight assigned to that study. The largest square belongs to the most precise study; the pooled estimate is shown as a diamond whose horizontal span equals the 95% confidence interval.

Fixed Effects and Random Effects Models

Before computing a pooled estimate, a fundamental question must be answered: do the included studies all estimate the same underlying effect, or does the true effect genuinely vary from study to study?

Fixed Effects Model

The fixed effects model assumes that all studies in the meta-analysis share a single common true effect $\theta$. Any observed differences between studies are attributed entirely to within-study sampling variation — random noise that would disappear if each study had infinite participants. Under this assumption, the inverse-variance weighted average described above is the optimal estimator.

The fixed effects model is appropriate when the studies are so similar in their populations, interventions, and outcome definitions that it is scientifically reasonable to assume they are measuring the identical effect. In practice this is often a strong assumption.

Random Effects Model

The random effects model, introduced by DerSimonian and Laird (1986), relaxes this assumption. It allows the true effect to vary across studies according to a distribution with mean $\mu$ and between-study variance $\tau^2$ (tau-squared). The pooled estimate now represents the mean of this distribution — the average effect across the population of studies, rather than a single common effect.

The weight assigned to each study is modified to include both within-study variance and the estimated between-study variance:

$$ w_i^* = \frac{1}{v_i + \hat{\tau}^2} $$

Because between-study variance $\hat{\tau}^2$ is added to every denominator, smaller studies lose less relative weight than under the fixed effects model. The random effects model therefore produces more conservative estimates — wider confidence intervals — than the fixed effects model when heterogeneity is present.

Which model to choose? When studies are heterogeneous (see below), the random effects model is generally preferred because it honestly acknowledges that the true effect is not identical in all settings. The fixed effects model is more appropriate for tightly controlled laboratory replications than for clinical trials conducted across different populations and healthcare systems.

The Forest Plot

The forest plot is the signature visualisation of a meta-analysis. It displays the effect estimate and confidence interval from each individual study as a horizontal line with a square marker at the centre, and the pooled result as a diamond at the bottom. The position of each element on the horizontal axis shows the direction and magnitude of the effect; the width of the confidence interval reflects the precision of the estimate.

Forest plot showing six studies with odds ratios and 95% confidence intervals. Each study is represented by a square (size proportional to weight) and a horizontal CI line. The pooled effect diamond at the bottom shows OR 1.68 (95% CI 1.35–2.09), favouring the exposed group.

Reading the forest plot above:

Heterogeneity

Even when all studies address the same research question, their effect estimates will rarely be identical. Some variation is expected from sampling alone. Heterogeneity refers to variation in the true underlying effects that goes beyond what sampling error alone can explain — systematic differences driven by patient populations, doses, follow-up durations, outcome definitions, or study quality. Assessing heterogeneity is essential before interpreting any meta-analysis.

Cochran's Q

Cochran's Q is the weighted sum of squared deviations of each study's estimate from the pooled weighted average:

$$ Q = \sum_{i=1}^{k} w_i\,(\theta_i - \bar{\theta}_w)^2 $$

Under the null hypothesis of homogeneity (all studies share the same true effect), $Q$ follows a chi-squared distribution with $k - 1$ degrees of freedom. A significant $Q$ — the conventional cutoff is $p < 0.10$ rather than the usual 0.05, because the test has low power when $k$ is small — suggests that genuine heterogeneity is present.

However, $Q$ alone is a poor guide to the magnitude of heterogeneity: with many studies it can be significant even when the degree of inconsistency is trivially small, and with few studies it may fail to detect substantial heterogeneity.

$I^2$ Statistic

The $I^2$ statistic addresses this limitation by expressing heterogeneity as the proportion of total variability in study estimates that is due to genuine between-study differences rather than sampling error:

$$ I^2 = 100\% \times \frac{Q - (k-1)}{Q} $$

$I^2$ is constrained to $[0\%, 100\%]$; negative values of the fraction are set to zero. The following ranges provide a rough guide to interpretation:

$I^2$Interpretation
0–40%Heterogeneity may not be important
30–60%May represent moderate heterogeneity
50–90%May represent substantial heterogeneity
75–100%Considerable heterogeneity

The ranges intentionally overlap because the importance of heterogeneity depends on context. An $I^2$ of 60% in a pharmacokinetic meta-analysis of identical compounds is more troubling than the same value in a meta-analysis of behavioural interventions across diverse populations, where variation is expected.

Example: A meta-analysis of six trials of a new antihypertensive agent yields $Q = 9.3$ on 5 degrees of freedom ($p = 0.097$, just significant at the 10% threshold). $I^2 = 100\% \times (9.3 - 5)/9.3 = 46\%$, suggesting moderate heterogeneity. The investigators switch from a fixed effects to a random effects model and investigate whether patient age or baseline blood pressure explains the variability between studies.

Publication Bias

A meta-analysis can only synthesise studies that have been published or otherwise made available. If studies with statistically significant positive results are more likely to be published than studies with null or negative results — a well-documented phenomenon — the available literature presents a biased picture of the true effect. This is publication bias, and it is a fundamental threat to the validity of any meta-analysis.

The Funnel Plot

The funnel plot is the primary visual tool for detecting publication bias. Each study is plotted as a point, with the effect estimate on the horizontal axis and the standard error (a measure of imprecision) on the vertical axis — with zero at the top and increasing values downward, so that precise large studies appear high up and imprecise small studies fall to the bottom.

Funnel plot showing six studies plotted as circles. The studies are roughly symmetrically distributed within the shaded funnel (pseudo-95% confidence limits). The red dashed vertical line marks the pooled estimate at OR 1.68.

In the absence of publication bias and heterogeneity, studies should be symmetrically distributed around the pooled effect within an inverted funnel shape: precise studies cluster near the top close to the true effect, while imprecise studies scatter more widely below. Asymmetry — particularly a gap at the bottom-left where small negative studies would appear — suggests that unfavourable results may have gone unpublished, inflating the apparent benefit.

In the example above the studies fall roughly symmetrically within the funnel, which is reassuring. However, funnel plots are unreliable with fewer than ten studies, and asymmetry can arise for reasons other than publication bias (e.g. genuine heterogeneity or poor methodology in smaller studies).

Egger's Test

Egger's test provides a formal statistical assessment of funnel plot asymmetry. It regresses the standardised effect estimate ($\theta_i / SE_i$, the precision-weighted effect) on the precision ($1/SE_i$). Under symmetry the regression line should pass through the origin; a significant non-zero intercept indicates asymmetry. A $p$-value below 0.05 is taken as evidence of potential publication bias.

Begg's Rank Test

Begg's rank correlation test assesses whether the rank correlation between the standardised effect sizes and their variances is significantly different from zero. Like Egger's test, a low $p$-value suggests potential publication bias. Both tests have limited power when $k$ is small, which is why visual inspection of the funnel plot remains important.

Types of Effect Measures

Meta-analysis can pool a wide range of effect measures, provided the same measure is used consistently across the included studies. MedCalc supports the following:

Effect measureTypical application
Mean difference or standardised mean differenceContinuous outcomes (e.g. blood pressure, pain scores) measured on the same or different scales
CorrelationAssociation between two continuous variables across studies
Odds ratioCase-control studies; logistic regression outputs
Relative riskCohort studies and randomised trials with binary outcomes
Risk differenceAbsolute effect size; used to compute NNT at the meta-analytic level
ProportionPrevalence or incidence across populations
Area under the ROC curveDiagnostic accuracy across studies of the same test
Generic inverse varianceAny effect measure for which the estimate and its standard error are available

The generic inverse variance method is especially useful when pooling hazard ratios from survival analyses or other measures that do not have dedicated meta-analysis routines, as long as the log-transformed estimate and its standard error can be extracted from each study.

MedCalc's meta-analysis procedures implement all the methods described on this page. For a full description of the available procedures and their output, see the the following sections of the MedCalc manual: