Comparison of partial areas under the ROC curve
Comparison of partial areas under ROC curve
When comparing 2 ROC curves, it may occur that the two Areas under the ROC curve (AUC) are equal, but one has a higher sensitivity than the other in a specificity range. The example shows two crossing ROC curves with equal total AUC. However, for the clinically important range (specificity higher than 80%), the sensitivity of test A is clearly higher than of test B (adapted from Obuchowski, 2006).
MedCalc allows to compare the two partial areas below the ROC curve in that specific interval. The data for the two partial areas can be derived from the same subjects (samples, patients, ...), in which case you have paired data; or from different subjects, in which case you have independent data and two independent partial areas.
See Partial area under ROC curve for detailed information on the calculation and interpretation of partial areas under the ROC curve.
- Variables: select the two variables of interest.
- Classification variable: select a dichotomous variable indicating diagnosis (0=negative, 1=positive).
If your data are coded differently, you can use the Define status tool to recode your data.
It is important to correctly identify the positive cases.
- Filter: (optionally) a filter in order to include only a selected subgroup of cases (e.g. AGE>21, SEX="Male").
- Data originate from the same subjects (paired data): select this option of the two variables contain data that originate from the same subjects. If the two datasets come from different subjects, then there is no paired design and you unselect this option. It is important to make this distinction because the statistical power of the test is different for a paired or independent samples model.
- Sensitivity or Specificity interval: select whether you want the partial area under the ROC curve for a defined sensitivy or specificity interval.
- From and To: the lower and upper value of the interval of interest. Common values are from 80%, 90%, 95% to 100%.
- Bootstrap Confidence Interval: select this option to calculate a confidence interval for the partial areas under the ROC curve, and their difference, using the bootstrap technique (Efron, 1987; Efron & Tibshirani, 1993).
- Advanced: click this button to specify the bootstrap parameters: number of replications and random number seed.
- Select Display ROC curve window to obtain the graph in a separate window.
Results - Paired samples
The following report is displayed in case of a paired design:
First MedCalc shows the parameters of the analysis.
Next, the following are reported for the two variables:
- Area under curve: this is the total area under the ROC curve.
- The Partial Area (pAUC): the area under the ROC curve in the specified specifity (or sensitivity) interval.
- The 95% Bootstrap CI of pAUC is reported if the corresponding option has been selected.
- The Standardized Partial Area (pAUCs) as defined by McCLish (1989), see above.
- The 95% Bootstrap CI of pAUCs is reported if the corresponding option has been selected. If this confidence interval does not include 0.5, then it can be concluded that the pAUCs is significantly different from the random model.
Results - Independent samples
The report for independent samples is somewhat different, but essentially contains the same statistics:
Comparison of the two partial areas
MedCalc reports the diffence between the two partial areas with the 95% bootstrap confidence interval for the difference. If the confidence interval does not include 0, then it can be concluded that the two partial areas are significantly different (P<0.05).
The ROC curve and partial areas are displayed in a separate window:
- Efron B (1987) Better Bootstrap Confidence Intervals. Journal of the American Statistical Association 82:171-185.
- Efron B, Tibshirani RJ (1993) An introduction to the Bootstrap. Chapman & Hall/CRC.
- McClish DK (1989) Analyzing a Portion of the ROC Curve. Medical Decision Making 9:190-195
- Obuchowski NA (2006) Receiver Operating Characteristic Curves and their use in radiology. Radiology 229:3-8.
- Zhou XH, Obuchowski NA, McClish DK (2002) Statistical methods in diagnostic medicine. Wiley-Interscience.
Statistical Methods in Diagnostic Medicine
Xiao-Hua Zhou, Nancy A. Obuchowski, Donna K. McClish
Statistical Methods in Diagnostic Medicine provides a comprehensive approach to the topic, guiding readers through the necessary practices for understanding these studies and generalizing the results to patient populations.
Following a basic introduction to measuring test accuracy and study design, the authors successfully define various measures of diagnostic accuracy, describe strategies for designing diagnostic accuracy studies, and present key statistical methods for estimating and comparing test accuracy.