Independent samples t-test
The independent samples (or two-sample) t-test is used to compare the means of two independent samples.
Select the variables for sample 1 and sample 2. You can use the button to select variables and selection criteria in the variables list.
The results windows for the Independent samples t-test displays the summary statistics of the two samples, followed by the statistical tests.
First an F-test is performed. If the P-value is low (P<0.05) the variances of the two samples cannot be assumed to be equal and it should be considered to use the t-test with a correction for unequal variances (Welch test) (see above).
The independent samples t-test is used to test the hypothesis that the difference between the means of two samples is equal to 0 (this hypothesis is therefore called the null hypothesis). The program displays the difference between the two means, and the 95% Confidence Interval (CI) of this difference. Next follow the test statistic t, the Degrees of Freedom (DF) and the two-tailed probability P. When the P-value is less than the conventional 0.05, the null hypothesis is rejected and the conclusion is that the two means do indeed differ significantly.
If you selected the Log transformation option, the program performs the calculations on the logarithms of the observations, but reports the back-transformed summary statistics.
For the t-test, the difference and 95% confidence are given, and the test is performed, on the log-transformed scale.
Next, the results of the t-test are transformed back and the interpretation is as follows: the back-transformed difference of the means of the logs is the ratio of the geometric means of the two samples (see Bland, 2000).
One-sided or two-sided tests
In MedCalc, P-values are always two-sided (as recommended by Fleiss, 1981, and Altman, 1991) and not one-sided.
A two-sided (or two-tailed) P-value is appropriate when the difference between the two means can occur in both directions: it may be either negative or positive, the mean of one sample may either be smaller or larger than that of the other sample.
A one-sided test should only be performed when, before the start of the study, it has already been established that a difference can only occur in one direction. E.g. when the mean of sample A must be more than the mean of sample B for reasons other than those connected with the sample(s).
Interpretation of P-values
P-values should not be interpreted too strictly. Although a significance level of 5% is generally accepted as a cut-off point for a significant versus a non-significant result, it would be a mistake to interpret a shift of P-value from e.g. 0.045 to 0.055 as a change from significance to non-significance. Therefore the real P-values are preferably reported, P=0.045 or P=0.055, instead of P<0.05 or P>0.05, so the reader can make his own interpretation.
With regards to the interpretation of P-values as significant versus not-significant, is has been recommended to select a smaller significance level of for example 0.01 when it is necessary to be quite certain that a difference exists before accepting it. When a study is designed to uncover a difference, or when a life-saving drug is being studied, we should be willing to accept that there is a difference even when the P-value is as large as 0.10 or even 0.20 (Lentner, 1982). The latter authors state that "The tendency in medical and biological investigations is to use too small a significance probability".
Whereas the P-value may give information on the statistical significance of the result, the 95% confidence interval gives information to assess the clinical importance of the result.
When the number of cases included in the study is large, a biologically unimportant difference can be statistically highly significant. A statistically significant result does not necessarily indicate a real biological difference.
On the other hand, a high P-value can lead to the conclusion of statistically non-significant difference although the difference is clinically meaningful and relevant, especially when the number of cases is small. A non-significant result does not mean that there is no real biological difference.
Confidence intervals are therefore helpful in interpretation of a difference, whether or not it is statistically significant (Altman et al., 1983).
Presentation of results
It is recommended to report the results of the t-test (and other tests) not by a simple statement such as P<0.05, but by giving full statistical information, as in the following example by Gardner & Altman (1986):
The difference between the sample mean systolic blood pressure in diabetics and non-diabetics was 6.0 mm Hg, with a 95% confidence interval from 1.1 to 10.9 mm Hg; the t test statistic was 2.4, with 198 degrees of freedom and an associated P value of P=0.02.
Mean 6.0 mm Hg, 95% CI 1.1 to 10.9; t=2.4, df=198, P=0.02