Comparison of Two Groups of Independent Observations

Two groups of observations are independent when there is no systematic relationship between individual values in one group and individual values in the other — subjects are allocated or selected separately, and no matching or repeated measurement links them. This is the most common comparison structure in clinical research: a treated group versus a control group in a randomised trial, or patients with a disease versus healthy controls in a case-control study. When the data in both groups are approximately normally distributed, the two-sample t-test (also called the independent-samples or unpaired t-test) is the appropriate parametric method. When the normality assumption cannot be justified — particularly with small samples or markedly skewed distributions — the Mann–Whitney test (also known as the Wilcoxon rank-sum test) provides a robust non-parametric alternative. If each observation in one group is linked to exactly one observation in the other, the data are paired and the methods in the Two groups of paired observations chapter should be used instead.

The Two-Sample t-Test

Let the two groups have sizes $n_1$ and $n_2$, sample means $\bar{x}_1$ and $\bar{x}_2$, and sample standard deviations $s_1$ and $s_2$. The null hypothesis is $H_0: \mu_1 = \mu_2$, and inference focuses on the difference $\bar{x}_1 - \bar{x}_2$.

When the two population variances can be assumed equal, they are estimated by the pooled variance:

This is a weighted average of the two sample variances, with more weight given to the larger sample. The test statistic is:

Under $H_0$, $t$ follows a t-distribution with $n_1 + n_2 - 2$ degrees of freedom. The 95% confidence interval for the true difference in means is:

The test assumes that both groups are drawn from normally distributed populations with equal variances. Normality can be assessed with histograms, Normal Q-Q plots, or formal tests (see the Are the data normally distributed? chapter). The equal-variance assumption is dropped in Welch's variant described in the next section.

Welch's t-Test for Unequal Variances

When the two groups have different variances, pooling them is incorrect and the pooled test can be seriously misleading — particularly when group sizes are also unequal. Welch's t-test relaxes the equal-variance assumption by using a separate variance estimate for each group. The standard error of the difference becomes:

and the test statistic $t = (\bar{x}_1 - \bar{x}_2) / \text{SE}$ is referred to the t-distribution with degrees of freedom given by the Welch–Satterthwaite approximation:

The resulting $\nu$ is generally not an integer and is smaller than $n_1 + n_2 - 2$, reflecting the loss of efficiency when variances are estimated separately.

In practice, Welch's t-test is recommended as the default for two independent groups. It performs essentially identically to the pooled test when variances are equal, and substantially better when they are not. Using a preliminary test of variance equality (such as Levene's or the F-test) to decide between the two variants is not recommended: the pre-test introduces its own error and the two-stage procedure does not control the overall Type I error rate well. Defaulting to Welch avoids this problem entirely.

The Mann–Whitney Test

The Mann–Whitney test (equivalently, the Wilcoxon rank-sum test) is the non-parametric alternative for comparing two independent groups. It does not require the data to be normally distributed. The null hypothesis is that, for a randomly selected observation from each group, neither is more likely to be larger than the other — equivalently, that the two populations have the same distribution.

The procedure pools all $n_1 + n_2$ observations, ranks them from 1 (smallest) to $n_1 + n_2$ (largest), and computes $R_1$, the sum of ranks assigned to group 1. From this, the test statistic is:

$U_1$ counts the number of times an observation from group 1 exceeds an observation from group 2 across all $n_1 n_2$ possible pairs. The complementary count $U_2 = n_1 n_2 - U_1$ counts the reverse. Under $H_0$, the expected value of each is $n_1 n_2 / 2$. For larger samples, the normal approximation

is used, with a correction applied when ties are present.

The ratio $U_1 / (n_1 n_2)$ is a direct estimate of the probability that a randomly selected observation from group 1 exceeds a randomly selected observation from group 2 — the same quantity as the AUC in ROC analysis. A value near 0.5 indicates no tendency for one group to produce larger values than the other; values near 0 or 1 indicate strong separation between the groups.

The Mann–Whitney test is sensitive to any difference between the two distributions, not only a shift in location. If the two groups differ in spread or shape but have the same median, the test may still be significant. This broader sensitivity is a strength in many clinical settings but should be kept in mind when interpreting results.

Worked Example

A randomised controlled trial compares two oral hypoglycaemic agents in 30 patients with type 2 diabetes (15 per arm). The primary outcome is reduction in HbA1c (percentage points) after 12 weeks. Summary statistics are:

The SDs are moderately different (0.61 vs 0.84), so Welch's t-test is appropriate. The standard error of the difference is:

The test statistic is:

The Welch–Satterthwaite degrees of freedom work out to approximately 25. With $t_{0.025,\;25} = 2.060$, the 95% confidence interval for the difference in mean HbA1c reduction is:

The two-tailed $P = 0.025$. Drug A reduced HbA1c by a mean of 0.64 percentage points more than Drug B (95% CI 0.09 to 1.19 pp, $P = 0.025$). The lower end of the confidence interval, though positive, is close to zero and only marginally above what might be considered clinically important; a larger trial would be needed to establish whether the advantage of Drug A is clinically as well as statistically reliable.

MedCalc computes the two-sample t-test (both pooled and Welch variants) in its Comparison of two means procedure, and the Mann–Whitney test in its Mann–Whitney test procedure.