Comparison of partial areas under the ROC curve

Command:

Statistics ROC curves Comparison of partial areas under ROC curve

Introduction

When comparing 2 ROC curves, it may occur that the two Areas under the ROC curve (AUC) are equal, but one has a higher sensitivity than the other in a specificity range. The example shows two crossing ROC curves with equal total AUC. However, for the clinically important range (specificity higher than 80%), the sensitivity of test A is clearly higher than of test B (adapted from Obuchowski, 2006).

MedCalc allows to compare the two partial areas below the ROC curve in that specific interval. The data for the two partial areas can be derived from the same subjects (samples, patients, ...), in which case you have paired data; or from different subjects, in which case you have independent data and two independent partial areas.

See Partial area under ROC curve for detailed information on the calculation and interpretation of partial areas under the ROC curve.

Required input

• Variables: select the two variables of interest.
• Classification variable: select a dichotomous variable indicating diagnosis (0=negative, 1=positive).

  If your data are coded differently, you can use the Define status tool to recode your data.

  It is important to correctly identify the positive cases.

• Filter: (optionally) a filter in order to include only a selected subgroup of cases (e.g. AGE>21, SEX="Male").
• Options:
  • Data originate from the same subjects (paired data): select this option of the two variables contain data that originate from the same subjects. If the two datasets come from different subjects, then there is no paired design and you unselect this option. It is important to make this distinction because the statistical power of the test is different for a paired or independent samples model.
  • Sensitivity or Specificity interval: select whether you want the partial area under the ROC curve for a defined sensitivity or specificity interval.
  • From and To: the lower and upper value of the interval of interest. Common values are from 80%, 90%, 95% to 100%.
  • Bootstrap Confidence Interval: select this option to calculate a confidence interval for the partial areas under the ROC curve, and their difference, using the bootstrap technique (Efron, 1987; Efron & Tibshirani, 1993).
  • Advanced: click this button to specify the bootstrap parameters: number of replications and random number seed.
  • Select Display ROC curve window to obtain the graph in a separate window.

Results - Paired samples

The following report is displayed in case of a paired design:

First MedCalc shows the parameters of the analysis.

Next, the following are reported for the two variables:

• Area under curve: this is the total area under the ROC curve.
• The Partial Area (pAUC): the area under the ROC curve in the specified specifity (or sensitivity) interval.
• The 95% Bootstrap CI of pAUC is reported if the corresponding option has been selected.
• The Standardized Partial Area (pAUCs) as defined by McClish (1989), see Partial area under ROC curve.
• The 95% Bootstrap CI of pAUCs is reported if the corresponding option has been selected. If this confidence interval does not include 0.5, then it can be concluded that the pAUCs is significantly different from the random model.

Results - Independent samples

The report for independent samples is somewhat different, but essentially contains the same statistics:

Comparison of the two partial areas

MedCalc reports the difference between the two partial areas with the 95% bootstrap confidence interval for the difference. If the confidence interval does not include 0, then it can be concluded that the two partial areas are significantly different (P<0.05).

Graph

The ROC curve and partial areas are displayed in a separate window:

Literature

• Efron B (1987) Better Bootstrap Confidence Intervals. Journal of the American Statistical Association 82:171-185.
• Efron B, Tibshirani RJ (1993) An introduction to the Bootstrap. Chapman & Hall/CRC.
• McClish DK (1989) Analyzing a Portion of the ROC Curve. Medical Decision Making 9:190-195.
• Obuchowski NA (2006) Receiver Operating Characteristic Curves and their use in radiology. Radiology 229:3-8.
• Zhou XH, Obuchowski NA, McClish DK (2002) Statistical methods in diagnostic medicine. Wiley-Interscience.

See also

• ROC curve analysis
• Partial area under ROC curve
• Comparison of ROC curves (correlated ROC curves, derived from the same cases)
• Comparison of independent ROC curves (ROC curves of subgroups)
• Format graph
• Graph legend
• Add graphical objects
• Reference lines

External links

• Receiver operating characteristic on Wikipedia.

Recommended book

Statistical Methods in Diagnostic Medicine
Xiao-Hua Zhou, Nancy A. Obuchowski, Donna K. McClish

Buy from Amazon

Statistical Methods in Diagnostic Medicine provides a comprehensive approach to the topic, guiding readers through the necessary practices for understanding these studies and generalizing the results to patient populations. Following a basic introduction to measuring test accuracy and study design, the authors successfully define various measures of diagnostic accuracy, describe strategies for designing diagnostic accuracy studies, and present key statistical methods for estimating and comparing test accuracy.