Fisher's Exact Test

When two binary categorical variables are cross-tabulated in a 2×2 table, the question is usually whether the row variable and the column variable are associated — whether, for example, the proportion responding to a treatment differs between treated and untreated patients. The most commonly taught approach is the chi-squared test, which compares observed cell counts to expected counts under the null hypothesis of independence. But the chi-squared test is an approximation, and it can give misleading results when the table is sparse: when expected cell counts are small, the $\chi^2$ distribution is a poor model for the actual sampling distribution of the test statistic. Fisher's exact test avoids this problem entirely by computing the exact probability of the observed table — and every table more extreme — directly from the combinatorial properties of the data, without any distributional approximation.

Expected Counts and When to Use Fisher's Test

Given a 2×2 table with row marginals $n_1$ and $n_2$, column marginals $m_1$ and $m_2$, and grand total $n$, the expected count for the cell in row $i$ and column $j$ is:

The chi-squared approximation is adequate when all four expected cell counts are at least 5. When one or more expected counts fall below this threshold — which occurs routinely in pilot studies, rare disease research, laboratory experiments, and case series — Fisher's exact test should be used instead. In practice, many statisticians recommend Fisher's exact test as the default for any 2×2 table, since it is always valid and the chi-squared approximation adds nothing except computational simplicity, which is no longer a consideration with modern software.

The Hypergeometric Distribution

The exact calculation rests on the hypergeometric distribution. Consider a 2×2 table with fixed marginals:

Once the marginal totals are fixed, the entire table is determined by the single cell count $a$. The probability of observing exactly $a$ events in the exposed group, given the fixed marginals, is:

This is the hypergeometric probability: the number of ways to choose $a$ events from the $m_1$ total events, times the number of ways to choose the remaining $n_1 - a$ exposed subjects from the $m_2$ non-events, divided by the total number of ways to choose $n_1$ subjects from all $n$. The key point is that these probabilities are exact — they require no assumption about the distribution of the data, only that the marginal totals are treated as fixed.

The Exact P-value

Under the null hypothesis of independence, the cell count $a$ follows the hypergeometric distribution with parameters $n$, $m_1$, and $n_1$. The exact two-tailed $P$ value is the sum of the probabilities of all possible tables whose probability is less than or equal to the probability of the observed table:

In other words, the $P$ value counts the total probability of outcomes at least as unusual as the one observed. This is the standard method recommended by Fisher and remains the most widely used approach for two-tailed inference.

Worked Example

A pilot study evaluates a novel immunotherapy in 14 patients with a rare autoimmune condition. Seven patients are randomly allocated to immunotherapy and seven to placebo. After three months, response is assessed:

The expected count for each cell is $7 \times 7 / 14 = 3.5$, which is below 5. Fisher's exact test is required.

With the marginals fixed at $n_1 = n_2 = 7$ and $m_1 = m_2 = 7$, the number of responders in the immunotherapy group, $a$, can range from 0 to 7. The hypergeometric probabilities for each possible value of $a$ are, with $\binom{14}{7} = 3432$:

The observed table has $a = 6$, with $P(6) = 49/3432 \approx 0.0143$. The two-tailed $P$ value is the sum of all $P(a)$ that are less than or equal to $P(6) = 0.0143$:

The association is statistically significant at the conventional 5% level. Six of seven patients responded in the immunotherapy group compared with one of seven in the placebo group, and this disparity is unlikely to have arisen by chance under the null hypothesis of independence ($P = 0.029$).

One-Tailed versus Two-Tailed Testing

The one-tailed $P$ value — the probability of a result at least as extreme as observed in one specified direction — is simply the sum of probabilities in the tail containing the observed value. In the example, the one-tailed $P = (49+1)/3432 \approx 0.015$.

A one-tailed test is valid only when the direction of the association was specified unambiguously before the data were collected, and when a difference in the opposite direction would be interpreted identically to no difference at all. These conditions are rarely met in clinical research. The two-tailed test is appropriate as the default, and should be reported unless there is a pre-specified and scientifically compelling reason to use a one-tailed approach.

Extensions

Fisher's exact test generalises beyond the 2×2 table. For $r \times c$ contingency tables — for example, when comparing response rates across three or more treatment groups, or when both variables have more than two categories — exact tests based on the multivariate hypergeometric distribution are available. The computation is considerably more demanding, requiring enumeration of all tables with the same marginals, but is handled automatically by software. For large sparse tables with many cells, these computations can be slow; Monte Carlo methods that approximate the exact $P$ value by random sampling from the reference distribution are a practical alternative.

MedCalc computes Fisher's exact two-tailed $P$ value in its Fisher's exact test procedure.