Pearson's coefficient of correlation – Bivariate Statistics (Senior 5 Mathematics) | Ganzaa

Bivariate Statistics

Lessons

Senior 5 · Mathematics

Bivariate Statistics

4 lessons

Lesson 3 of 4

Pearson's coefficient of correlation

40 min

One number for "how strong"

Lesson 2 ended with a problem: the covariance of the revision data was 29 in hours and marks, but 3480 in minutes and percentages, although nothing about the pupils had changed. To compare the strength of different relationships, a measure must not depend on units. Karl Pearson's coefficient, introduced in the 1890s, divides the covariance by the two standard deviations, and the units cancel. It is the most widely used number in statistics for a linear relationship.

Definition

Pearson's (linear) coefficient of correlation of $X$ and $Y$ is

r = \frac{\operatorname{cov}(X, Y)}{\sigma_X\,\sigma_Y}, \qquad \text{where } \sigma_X = \sqrt{\operatorname{Var}(X)} = \sqrt{\frac{1}{n}\sum x_i^2 - \bar{x}^2} \text{ and similarly for } \sigma_Y .

Multiplying the numerator and denominator by $n^{2}$ gives a form that uses only the column totals:

r = \frac{n\sum xy - \sum x \sum y}{\sqrt{\left(n\sum x^2 - \left(\sum x\right)^2\right)\left(n\sum y^2 - \left(\sum y\right)^2\right)}} .

Important

Properties of $r$

$-1 \le r \le 1$ .
$r = 1$ or $r = - 1$ exactly when all the points lie on one straight line (rising or falling).
$r$ has no units, and does not change when either variable is shifted or multiplied by a positive number: $r(aX + b,\ cY + d) = r(X, Y)$ if $a c > 0$ . If $a c < 0$ , only the sign changes.
The sign of $r$ is the sign of the covariance: the direction of the trend.
$r$ measures linear association only. $r \approx 0$ means "no linear trend", not "no relationship".

Reason for property 3. By lesson 2, $\operatorname{cov}(aX + b, cY + d) = ac\operatorname{cov}(X, Y)$ , while $\sigma_{aX + b} = |a|\sigma_X$ and $\sigma_{cY + d} = |c|\sigma_Y$ . The factors cancel except for the sign of $a c$ . Property 1 follows from the Cauchy–Schwarz inequality $|u \cdot v| \le |u|\,|v|$ of unit 7, applied to the vectors of deviations $(x_i - \bar{x})$ and $(y_i - \bar{y})$ : $r$ is the cosine of the angle between them.

Reading the value of $r$

The bands are a common guide, not a law: in a physics experiment $r = 0.8$ may be disappointing, while in a survey of people it may be remarkable. Always look at the scatter diagram as well.

Worked example 1: $r$ for the revision data

Solution. From lesson 2: $\operatorname{cov}(X, Y) = 29$ , $\operatorname{Var}(X) = 11.5$ , $\operatorname{Var}(Y) = 77$ . So

r = \frac{29}{\sqrt{11.5}\,\sqrt{77}} = \frac{29}{3.391 \times 8.775} \approx 0.97 .

A strong positive linear correlation, as the scatter diagram suggested.

Worked example 2: using the column totals

The age $x$ (years) and resale value $y$ (hundred thousand FRW) of five motorcycles are $(4, 30), (6, 26), (8, 25), (10, 20), (12, 14)$ (illustrative). Find $r$ .

Solution. $n = 5$ , $\sum x = 40$ , $\sum y = 115$ , $\sum x^2 = 360$ , $\sum y^2 = 2797$ , $\sum xy = 844$ .

r = \frac{5(844) - 40(115)}{\sqrt{\left(5(360) - 40^2\right)\left(5(2797) - 115^2\right)}} = \frac{4220 - 4600}{\sqrt{200 \times 760}} = \frac{-380}{389.9} \approx -0.97 .

A strong negative correlation: older motorcycles are worth less.

Worked example 3: the households

For the 40 households of lesson 2, $\operatorname{cov}(X, Y) = 0.405$ , $\operatorname{Var}(X) = \frac{560}{40} - 3.6^2 = 1.04$ and $\operatorname{Var}(Y) = \frac{174}{40} - 1.95^2 = 0.5475$ . So $r = \frac{0.405}{\sqrt{1.04 \times 0.5475}} \approx 0.54$ : a moderate positive correlation. Bigger households tend to own more phones, but with many exceptions.

Worked example 4: a perfect relationship with $r = 0$

Find $r$ for the points $(- 2, 4), (- 1, 1), (0, 0), (1, 1), (2, 4)$ .

Solution. $\bar{x} = 0$ , $\bar{y} = 2$ , $\sum xy = -8 - 1 + 0 + 1 + 8 = 0$ , so $\operatorname{cov}(X, Y) = 0 - 0 \times 2 = 0$ and $r = 0$ . Yet every point lies exactly on $y = x^{2}$ . The relationship is perfect but not linear: the falling left half and rising right half cancel.

Warning

Correlation is not causation, however close $r$ is to 1.
One outlier can change $r$ a lot, in either direction. Look at the scatter diagram before trusting $r$ .
A value outside $[- 1, 1]$ is always an arithmetic error. Check the totals.
Do not round intermediate values too early; keep at least four significant figures until the end.

Tip

The numerator of the column-totals formula has the same sign as the covariance. Check its sign against the scatter diagram before finishing the calculation.

StatQuest with Josh Starmer: Pearson's Correlation, Clearly Explained (about 20 min). Watch how r measures how close the points are to a line, and why it says nothing about the line's slope.

✦

Practice

Questions 1 to 3 are routine; 4 to 6 need more care; 7 asks for a judgement.

Find $r$ for $(1, 2), (2, 4), (3, 5), (4, 4), (5, 7)$ (their covariance is 2, from lesson 2).
$\operatorname{cov}(X, Y) = -12$ , $\sigma_X = 4$ , $\sigma_Y = 5$ . Find $r$ and describe the correlation.
Describe in words the correlations $r = 0.93$ , $r = - 0.62$ and $r = 0.08$ .
$r (X, Y) = 0.8$ . Find $r(-2X + 3,\ 5Y)$ and $r(10X,\ Y - 7)$ .
For ten pairs, $\sum x = 50$ , $\sum y = 60$ , $\sum x^2 = 290$ , $\sum y^2 = 400$ , $\sum xy = 320$ . Find $r$ .
A pupil computes $r = 1.12$ . Without seeing the data, what can you say?
A newspaper reports $r = 0.9$ between the number of televisions sold in a country each year and the number of road accidents. Write two sentences judging the claim "televisions cause accidents".

Show answersHide answers

$\operatorname{Var}(X) = 2$ , $\operatorname{Var}(Y) = \frac{110}{5} - 4.4^2 = 2.64$ ; $r = \frac{2}{\sqrt{2 \times 2.64}} \approx 0.87$ .
$r = \frac{-12}{20} = -0.6$ : moderate negative.
Strong positive; moderate negative; weak or no linear correlation.
$- 0.8$ (one factor is negative); $0.8$ .
$r = \frac{10(320) - 50(60)}{\sqrt{(2900 - 2500)(4000 - 3600)}} = \frac{200}{400} = 0.5$ .
It is wrong: $r$ always lies between $- 1$ and 1, so there is an arithmetic error.
Both numbers may simply grow with time, population and wealth (more people, more televisions, more vehicles). A strong correlation alone cannot show that one causes the other.

Sources and credits

Written by Ganzaa: the explanations, the scale of r and its scatter diagrams (generated so that each has exactly the stated r), the worked examples, the practice items and the quiz questions. The data are illustrative. Every coefficient was checked by computer.
Video: Pearson's Correlation, Clearly Explained!!!, StatQuest with Josh Starmer, embedded from YouTube.

1One number for "how strong"

2Reading the value of r r r

One number for "how strong"

Reading the value of $r$