Method comparison: Comparison of multiple methods

Command: Statistics
Next selectMethod comparison & evaluation
Next selectComparison of multiple methods

Description

Comparison of multiple methods is an extension of the Bland-Altman plot (Bland & Altman, 1986 and 1999) for more than two methods. For each method, the differences with a reference method are plotted against the values of this reference method (Krouwer, 2008).

The procedure produces multiple bias plots in one single display with all axes aligned to facilitate comparison of the different methods.

If duplicate or multiple measurements (with two methods) were performed per subject, you should use Bland-Altman plot with multiple measurements per subject.

Required input

Dialog box for multiple method comparison

Select the variables for the methods you want to compare.

Options

  • First method is the reference method: the measurements of first selected method are the reference values (informational, this option is fixed).
  • Plot differences or ratios
    • Plot differences: Differences are calculated as measurement‑reference so a positive difference is an overestimation and a negative difference is an underestimation.
    • Plot differences as %: With this option the differences will be expressed as percentages of the values on the axis (i.e. proportionally to the magnitude of measurements). This option is useful when there is an increase in variability of the differences as the magnitude of the measurement increases.
    • Plot ratios: When this option is selected then the ratios of the measurements will be plotted instead of the differences (avoiding the need for logarithmic transformation). This option as well is useful when there is an increase in variability of the differences as the magnitude of the measurement increases. Ratios are calculated as measurement/reference so a ratio > 1 indicates an overestimation and a ratio < 1 indicates an underestimation.
  • Draw line of equality: useful for detecting a systematic difference.
  • Draw lines for 95% CI of mean of differences*: the 95% Confidence Interval of the mean difference illustrates the magnitude of the systematic difference. If the line of equality is not in the interval, there is a significant systematic difference.
  • Draw lines for 95% CI of limits of agreement: shows lines for the 95% confidence interval for both the upper and lower limits of agreement.
  • Draw regression line of differences*: this regression line may help to detect a proportional difference. Optionally, you can select to show the 95% confidence interval of this regression line.

*or ratios when this option was selected.

Results

The results panel displays the following information:

Results for multiple method comparison

  • Identifier for the reference value: the variable for the reference method.
  • Systematic differences: n (sample size), mean, SD and 95% CI of the differences.
  • Limits of agreement: the lower and upper limits of agreement with 95% CI.
  • Parameters of the regression of the differences against the reference value: intercept and slope with 95% CI, and P-value for slope.
  • Absolute percentage error: the absolute percentage error (APE) is calculated as 100 x ABS[(yref)/ref] where y is the observation and ref is the reference value. MedCalc calculates the median APE (MdAPE) and the 95th percentile of the absolute percentage error. The 95th percentile APE is interpreted as follows: the percentage difference between a measurement and the reference value is not expected, with 95% certainty, to exceed this value. MedCalc also reports the 95% confidence intervals for both statistics, if sample size is large enough.

Graph

The graphical display consists of multiple frames with Bland & Altman plots using the selected options.

Multiple method comparison graph (Bland-Altman plots)

Unlike other MedCalc graphs, this graphical display has limited editing possibilities:

  • you cannot add reference lines or draw boxes, arrows or add text frames.
  • the scaling of the axes is the same in each frame.
  • the titles are the same for each frame. To create variations in the titles for the different frames, you can use the following symbolic fields:
    • !#!: this will insert the frame number in the title
    • !a!: this will insert a character A, B, ..., corresponding with the frame number, in the title
    • !var!: this will insert the variable name in the title

Literature

  • Bland JM, Altman DG (1986) Statistical method for assessing agreement between two methods of clinical measurement. The Lancet i:307-310. PubMed
  • Bland JM, Altman DG (1995) Comparing methods of measurement: why plotting difference against standard method is misleading. The Lancet 346:1085-1087. PubMed
  • Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Statistical Methods in Medical Research 8:135-160. PubMed
  • Hanneman SK (2008) Design, analysis, and interpretation of method-comparison studies. AACN Advanced Critical Care 19:223-234. PubMed
  • Krouwer JS (2008) Why Bland-Altman plots should use X, not (Y+X)/2 when X is a reference method. Statistics in Medicine 27:778-780. PubMed

See also