Regression
Where correlation measures the of an association between two variables, regression goes further: it defines the of the curve that best describes the relationship, enabling prediction of one variable from the other. Given a value of the independent variable $x$, the regression equation gives the expected value of the dependent variable $y$.
An important distinction from correlation is that regression is directional. Correlation is symmetric — the correlation of $x$ with $y$ equals the correlation of $y$ with $x$. Regression is not: the equation for predicting blood pressure from age is a different line than the equation for predicting age from blood pressure. The choice of which variable is the outcome ($y$) and which is the predictor ($x$) must be made on scientific grounds before the analysis begins, not after looking at the data.
Scatter Diagram with Regression Line
As with any analysis of two variables, the first step is to inspect a scatter diagram. Plotting the data reveals the overall shape of the relationship — linear, curved, or absent — before any equation is fitted.
MedCalc can produce scatter diagrams with a fitted regression line using its Scatter diagram with regression line procedure.
Simple Linear Regression
The simplest model assumes a straight-line relationship between $x$ and $y$:
$$ \hat{y} = a + b\,x $$where $\hat{y}$ is the predicted value of $y$, $a$ is the intercept (the predicted value of $y$ when $x = 0$), and $b$ is the slope (the change in $\hat{y}$ for each unit increase in $x$).
Least-Squares Estimation
The coefficients $a$ and $b$ are chosen to minimise the sum of squared vertical distances between the observed values $y_i$ and the corresponding fitted values $\hat{y}_i$ — the least-squares criterion. These vertical distances are called residuals: $e_i = y_i - \hat{y}_i$. A residual is positive when the observed value lies above the line and negative when it lies below. Squaring the residuals before summing ensures that large deviations are penalised more heavily than small ones, and prevents positive and negative residuals from cancelling each other out. The resulting formulas are:
$$ b = \frac{\displaystyle\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\displaystyle\sum_{i=1}^{n}(x_i - \bar{x})^2} \qquad a = \bar{y} - b\,\bar{x} $$Note that the numerator of $b$ is identical to the numerator of the Pearson correlation coefficient — the two statistics are closely related.
Coefficient of Determination ($R^2$)
$R^2$ expresses the proportion of the total variation in $y$ that is explained by the regression on $x$:
$$ R^2 = 1 - \frac{\displaystyle\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}{\displaystyle\sum_{i=1}^{n}(y_i - \bar{y})^2} $$$R^2$ ranges from 0 (the model explains none of the variation) to 1 (a perfect fit). For simple linear regression, $R^2$ equals the square of the Pearson correlation coefficient $r$. A regression with $R^2 = 0.72$ means that 72% of the variability in $y$ is accounted for by the linear relationship with $x$.
Statistical Significance
The null hypothesis $H_0: \beta = 0$ (the slope is zero — no linear relationship) is tested with a $t$-statistic:
$$ t = \frac{b}{\text{SE}(b)}, \qquad \text{SE}(b) = \frac{s}{\sqrt{\displaystyle\sum_{i=1}^{n}(x_i - \bar{x})^2}} $$where $s = \sqrt{\frac{1}{n-2}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}$ is the residual standard deviation. Under $H_0$, the statistic follows a $t$-distribution with $n - 2$ degrees of freedom. MedCalc also reports a confidence interval for $b$.
Example: In 23 patients the least-squares fit gives $\hat{y} = 72.6 + 1.25\,x$, where $x$ is age in years and $\hat{y}$ is predicted systolic blood pressure in mmHg. The slope $b = 1.25$ means that, on average, systolic blood pressure increases by 1.25 mmHg for each additional year of age. With $R^2 = 0.73$, age explains 73% of the variability in blood pressure in this sample.
Other Regression Models
Not all relationships are linear. MedCalc fits five models; choosing the right one is guided by the shape of the scatter diagram and subject-matter knowledge.
| Model | Equation | Typical application |
|---|---|---|
| Straight line | $y = a + b\,x$ | Constant rate of change; the default starting point. |
| Logarithmic curve | $y = a + b\,\log(x)$ | Rapid early rise that levels off; e.g. drug effect vs. log-dose. |
| Exponential curve | $\log(y) = a + b\,x$ | Proportional growth or decay; e.g. bacterial counts over time. |
| Geometric (power) curve | $\log(y) = a + b\,\log(x)$ | Allometric relationships; e.g. organ weight vs. body weight. |
| Quadratic (parabola) | $y = a + b\,x + c\,x^2$ | U-shaped or inverted-U relationships; e.g. optimal dosing. |
The logarithmic, exponential, and geometric models are fitted by applying a logarithmic transformation to linearise the relationship, then using ordinary least squares on the transformed scale. This is mathematically convenient, but it requires care when interpreting results: the coefficients and $R^2$ describe the fit on the scale. For the exponential model, for instance, the fitted equation $\log(\hat{y}) = a + b\,x$ must be back-transformed to $\hat{y} = e^a \cdot e^{b\,x}$ to express predictions in the original units of $y$. The coefficient $e^b$ represents the multiplicative factor by which $y$ changes for each unit increase in $x$.
Choosing a Model
- Inspect the scatter diagram first. A straight band of points suggests a linear model; a bend suggests a curve.
- Compare $R^2$ values. A higher $R^2$ indicates a better-fitting model, though comparing models with different numbers of parameters requires care.
- Consider the biology. A model that fits better statistically but makes no biological sense should be treated with caution.
- Avoid extrapolation. Any regression equation is reliable only within the range of the observed data; predictions far beyond that range are speculative.
MedCalc fits all five models and reports $R^2$, the regression coefficients with confidence intervals, and residual plots in its Regression procedure.
Ordinary Regression Is Not Suitable for Method Comparison
It may seem natural to compare two measurement methods by regressing one against the other and checking whether the slope equals 1 and the intercept equals 0. This approach is incorrect.
Ordinary least-squares regression assumes that the $x$-variable is measured without error and that all measurement uncertainty lies in $y$. In method comparison, variables are measurements subject to error. Applying OLS under these conditions produces biased slope estimates — and the direction of the bias reverses depending on which method is placed on the $x$-axis. Two analysts using the same dataset can reach opposite conclusions simply by swapping the axes, which is unacceptable for a method comparison study.
Furthermore, even a perfect regression fit ($R^2 = 1$, slope $= 1$, intercept $= 0$) does not tell a clinician whether the two methods agree in practice — for that, the absolute differences between the measurements must be assessed against a pre-specified clinical acceptance limit.
The correct primary tool for method comparison is the Bland–Altman plot, which directly characterises the bias and limits of agreement in the original measurement units. When a regression model for the method relationship is specifically needed — for instance to quantify proportional bias — Passing–Bablok regression is the appropriate choice, as it handles measurement error in both variables correctly.