Correlation & Linear Regression

Scatter two variables against each other and a story often appears: taller people tend to be heavier, more revision tends to mean higher marks. Correlation measures how tight that story is, and regression draws the best straight line through it.

What correlation measures

Correlation captures the strength and direction of a straight-line relationship between two variables. We summarise it in a single number, the correlation coefficient r, which always lands between −1 and +1.

The sign tells you the direction: positive r means y tends to rise as x rises; negative r means y tends to fall. The size tells you the strength: the closer |r| is to 1, the more tightly the points hug a straight line. An r of exactly ±1 means the points lie perfectly on a line; an r of 0 means no linear pattern at all.

r measures how “line-like” a cloud of points is — not how steep the line is.

See it: fit a line

Drag the points below and watch r respond in real time. Pull the cloud into a tight upward band and r races towards +1; scatter them into a shapeless blob and r collapses towards 0. Notice too that the gold line always pivots through the mean point (x̄, ȳ).

The least-squares line

Of all the straight lines you could draw through a scatter, which is “best”? The regression line of y on x is defined as the one that makes the total squared vertical distance from the points to the line as small as possible. Those vertical gaps are called residuals— the error between each real y and the line’s prediction ŷ.

A useful fact falls out of the algebra: the least-squares line always passes through the mean point (x̄, ȳ). So even before you compute a gradient, you know one point the line must go through — a handy check.

Reading r — and its traps

Interpreting r sensibly is where marks are won and lost. Values near ±1 indicate a strong linear relationship, values near 0 a weak one or none. But r is a summary, and summaries hide things.

A single outliercan drag both the line and r a long way, flattering or wrecking an otherwise clear pattern — drag one point far off in the widget above and watch r lurch. And most important of all: a strong r tells you two variables move together, not that one causes the other. Correlation is not causation.

Common mistakes

Practice

Where next?

The natural sequel is r², the coefficient of determination: square the correlation and you get the fraction of the variation in y that the line actually explains.