Logistic Regression

Many clinical outcomes are binary: a patient either has a disease or does not, a treatment either succeeds or fails, a tumour is either malignant or benign. When the outcome variable takes only two values — conventionally coded 1 (event present) and 0 (event absent) — ordinary linear regression is inappropriate, because it can predict probabilities outside the range [0, 1]. Logistic regression solves this by modelling the probability of the outcome on a transformed scale that is unbounded.

The Logistic Model

Let $p$ denote the probability that the outcome is present. Instead of modelling $p$ directly, logistic regression models the log odds, also called the logit:

$$ \text{logit}(p) = \ln\!\left(\frac{p}{1-p}\right) = b_0 + b_1 x_1 + b_2 x_2 + \cdots + b_k x_k $$

The logit can take any value from $-\infty$ to $+\infty$, making it suitable for a linear predictor. Solving back for $p$ gives the logistic function (sigmoid curve):

$$ p = \frac{1}{1 + e^{-(b_0 + b_1 x_1 + \cdots + b_k x_k)}} $$

This S-shaped curve maps any value of the linear predictor to a probability strictly between 0 and 1.

Logistic regression curve showing the probability of hypertension as a function of age. The S-shaped red curve rises from near zero at age 20 to near one at age 80, crossing p = 0.5 at approximately age 53. Filled circles mark hypertensive patients and open circles normotensive patients.

The diagram illustrates a model with a single continuous predictor (age). In practice, logistic regression handles multiple predictors simultaneously — both continuous and categorical.

Coefficients and Odds Ratios

The coefficients $b_1, b_2, \ldots, b_k$ are estimated by maximum likelihood: the algorithm finds the values that make the observed outcomes most probable under the model.

Each coefficient $b_i$ represents the change in the log odds of the outcome for a one-unit increase in $x_i$, holding all other predictors constant. Because log odds are difficult to interpret directly, the key reported quantity is the odds ratio:

$$ \text{OR}_i = e^{b_i} $$

$\text{OR} > 1$: the odds of the outcome $increase$ with a one-unit increase in $x_i$.
$\text{OR} < 1$: the odds $decrease$ .
$\text{OR} = 1$: no association.

MedCalc reports each odds ratio together with its 95% confidence interval and a Wald test $p$-value. A confidence interval that excludes 1 indicates a statistically significant association.

Example: In a study of 350 adults, logistic regression with age and BMI as predictors yields the following coefficients: $b_{\text{age}} = 0.14$ and $b_{\text{BMI}} = 0.18$. The odds ratios are $e^{0.14} = 1.15$ and $e^{0.18} = 1.20$. For every additional year of age the odds of hypertension increase by 15%; for every additional unit of BMI they increase by 20%, after adjusting for the other predictor.

Assessing Model Fit

Overall Model Chi-squared

The overall significance of the model is tested by comparing the log likelihood of the fitted model against the null model (intercept only). The test statistic is:

$$ \chi^2 = -2\ln L_0 - (-2\ln L_1) $$

where $L_0$ and $L_1$ are the likelihoods of the null and fitted models respectively. Under the null hypothesis that all coefficients are zero, this statistic follows a chi-squared distribution with $k$ degrees of freedom (one per predictor).

Pseudo $R^2$

Unlike linear regression there is no exact equivalent of $R^2$ in logistic regression. Two commonly reported approximations are:

Cox & Snell $R^2$: based on the likelihood ratio, but has a maximum below 1 in most practical situations.
Nagelkerke $R^2$: a rescaled version of Cox & Snell $R^2$ that can reach 1; generally the preferred summary measure.

These values give a rough indication of the proportion of variation in the outcome explained by the model.

Hosmer–Lemeshow Test

The Hosmer–Lemeshow goodness-of-fit test divides observations into groups based on their predicted probabilities and compares predicted with observed event counts using a chi-squared statistic. A $non-significant$ result ($p > 0.05$) indicates that predicted and observed frequencies agree adequately — in other words, the model fits the data well. A significant result signals poor fit.

Classification Table

For each observation, the model produces a predicted probability. Applying a cut-off (typically 0.5) converts this to a predicted class (0 or 1). The classification table (confusion matrix) cross-tabulates predicted classes against observed outcomes, yielding:

Sensitivity: the proportion of true events correctly predicted as events.
Specificity: the proportion of true non-events correctly predicted as non-events.
Overall correct classification rate.

MedCalc can also produce a ROC curve from the predicted probabilities, allowing the choice of cut-off to be optimised.

Sample Size Requirements

Logistic regression requires adequate sample sizes to produce stable coefficient estimates. A widely used minimum is:

$$ n \geq \frac{10\,k}{p_{\min}} $$

where $k$ is the number of predictors and $p_{\min}$ is the proportion of the least frequent outcome category. For example, with 5 predictors and an event rate of 20% ($p_{\min} = 0.20$), the minimum sample size is $10 \times 5 / 0.20 = 250$. An absolute minimum of 100 observations is also recommended regardless of this formula.

MedCalc's Logistic regression procedure fits the model, reports coefficients, odds ratios with confidence intervals, model fit statistics, the Hosmer–Lemeshow test, and the classification table.