Skip to main content
MedCalc
Mail a PDF copy of this page to:
(Your email address will not be added to a mailing list)
working
Show menu Show menu

Comparison of independent ROC curves

Description

Use Comparison of independent ROC curves to compare the areas under the ROC curve between subgroups of cases (e.g. male - female). ROC curves are computed for each subgroup and the Areas under the ROC curve are compared pairwise.

Required input

Dialog box for comparison of independent ROC curves

Data

  • Variable: the continuous variable of interest.
  • Grouping variable: a categorical variable that defines subgroups. ROC curves will be computed for each subgroup.
  • Classification variable: a dichotomous variable indicating diagnosis (0=negative, 1=positive).

    If your data are coded differently, you can use the Define status tool to recode your data.
  • Filter: (optionally) a filter in order to include only a selected group of cases (e.g. AGE>21).

Methodology:

  • DeLong et al.: use the method of Delong et al. (1988) for the calculation of the Standard Error of the Area Under the Curve (AUC) (recommended).
  • Hanley & McNeil: use the methods of Hanley & McNeil (1982, 1983) for the calculation of the Standard Error of the Area Under the Curve (AUC).
  • Binomial exact Confidence Interval for the AUC: calculate exact Binomial Confidence Intervals for the Area Under the Curves (AUC) (recommended). If this option is not selected, the Confidence Intervals for the AUCs are calculated as AUC ± 1.96 SE (Standard Error). This option does not apply to the difference between two AUCs.

Graph

  • Select Display ROC curves window to obtain the ROC plots in a separate graph window.

    Option:
    • mark points corresponding to criterion values.

Results

Comparison of independent ROC curves - statistics

The results window for Comparison of independent ROC curves displays:

  1. The results for all subgroups
    • Area under the ROC curve (AUC), with standard error: this value can be interpreted as follows: an area of 0.84, for example, means that a randomly selected individual from the positive group has a test value larger than that for a randomly chosen individual from the negative group in 84% of the time (Zweig & Campbell, 1993). When the variable under study cannot distinguish between the two groups, i.e. where there is no difference between the two distributions, the area will be equal to 0.5 (the ROC curve will coincide with the diagonal). When there is a perfect separation of the values of the two groups, i.e. there is no overlapping of the distributions, the area under the ROC curve equals 1 (the ROC curve will reach the upper left corner of the plot).
    • 95% confidence interval: the 95% confidence interval for the area can be used to test the hypothesis that the theoretical area is 0.5. If the confidence interval does not include the 0.5 value, then there is evidence that the laboratory test does have an ability to distinguish between the two groups (Hanley & McNeil, 1982; Zweig & Campbell, 1993).
  2. Pairwise comparison of the Area under the curve for all subgroups
    • Difference between the areas with standard error: the difference between the two areas under the curve.
    • 95% confidence interval: the 95% confidence interval for this difference.
    • Significance level: probability of the hypothesis that the difference between the two AUCs is 0.

    Graph

    This graph displays the different ROC curves, allowing visual comparison.

    Comparison of independent ROC curves - graph

    In a ROC curve the true positive rate (Sensitivity) is plotted in function of the false positive rate (100-Specificity) for different cut-off points. Each point on the ROC plot represents a sensitivity/specificity pair corresponding to a particular decision threshold. A test with perfect discrimination (no overlap in the two distributions) has a ROC plot that passes through the upper left corner (100% sensitivity, 100% specificity). Therefore the closer the ROC plot is to the upper left corner, the higher the overall accuracy of the test (Zweig & Campbell, 1993).

    Literature

    • DeLong ER, DeLong DM, Clarke-Pearson DL (1988): Comparing the areas under two or more correlated receiver operating characteristic curves: a non-parametric approach. Biometrics 44:837-845.
    • Hanley JA, Hajian-Tilaki KO (1997) Sampling variability of non-parametric estimates of the areas under receiver operating characteristic curves: an update. Academic Rediology 4:49-58.
    • Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29-36.
    • Hanley JA, McNeil BJ (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839-843.
    • Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry 39:561-577.

    See also