Two-Variable Statistics

Lessons

Senior 6 · Basic Mathematics

Two-Variable Statistics

4 lessons

Lesson 2 of 4

Covariance and Pearson's coefficient of correlation

25 min

From a picture to a number

"Strong" and "weak" are judgements; two people can look at the same scatter diagram and disagree. To compare data sets fairly, statisticians measure correlation with a number. The measure is built in two steps: first the covariance, then Pearson's coefficient of correlation, which rescales the covariance so that it always lies between $- 1$ and $1$ .

A kiosk owner in Kigali counted the bottles of water she sold on five days and noted each day's maximum temperature (illustrative data):

Day	1	2	3	4	5
Temperature, $x$ (°C)	22	24	25	27	29
Bottles sold, $y$	40	46	50	55	62

Review: mean and standard deviation

For $n$ values $x_1, x_2, \ldots, x_n$ :

\bar{x} = \frac{\sum x_i}{n}, \qquad \sigma_x^2 = \frac{\sum (x_i - \bar{x})^2}{n} = \frac{\sum x_i^2}{n} - \bar{x}^2, \qquad \sigma_x = \sqrt{\sigma_x^2}

The variance $\sigma_x^2$ is the mean of the squared distances from the mean; the standard deviation $\sigma_x$ brings it back to the original units. The second form, "mean of the squares minus the square of the mean", is quicker with a table of $x^{2}$ .

Why the covariance works

Draw a vertical line at $\bar{x}$ and a horizontal line at $\bar{y}$ . For each point form the product $(x_i - \bar{x})(y_i - \bar{y})$ .

A point above and to the right of the means gives (positive)(positive), a positive product; a point below and to the left gives (negative)(negative), also positive. Points in the other two quadrants give negative products. When most points lie in the "+" quadrants, the sum of the products is positive: positive correlation. Here all five points lie in the "+" quadrants.

Definition

The covariance of $x$ and $y$ is the mean of the products of the deviations:

\operatorname{cov}(x, y) = \frac{1}{n}\sum (x_i - \bar{x})(y_i - \bar{y}) = \frac{\sum x_i y_i}{n} - \bar{x}\,\bar{y}

Its sign gives the direction of the linear relationship. Its size depends on the units, so on its own it cannot say "strong" or "weak". Example: for the pairs $(1, 2)$ , $(2, 4)$ , $(3, 6)$ , $\operatorname{cov} = \frac{2 + 8 + 18}{3} - 2 \times 4 = \frac{4}{3}$ .

Definition

Pearson's coefficient of correlation $r$ divides the covariance by both standard deviations:

r = \frac{\operatorname{cov}(x, y)}{\sigma_x\,\sigma_y}

It has no units. Example: if $\operatorname{cov}(x, y) = 12$ , $\sigma_x = 4$ and $\sigma_y = 5$ , then $r = \frac{12}{20} = 0.6$ .

Worked example

Worked example 1. Find the covariance and $r$ for the water data.

$x$	$y$	$x^{2}$	$y^{2}$	$x y$
22	40	484	1600	880
24	46	576	2116	1104
25	50	625	2500	1250
27	55	729	3025	1485
29	62	841	3844	1798
127	253	3255	13085	6517

Means: $\bar{x} = \frac{127}{5} = 25.4$ , $\bar{y} = \frac{253}{5} = 50.6$ .

\operatorname{cov}(x, y) = \frac{6517}{5} - 25.4 \times 50.6 = 1303.4 - 1285.24 = 18.16

\sigma_x^2 = \frac{3255}{5} - 25.4^2 = 651 - 645.16 = 5.84, \quad \sigma_x = 2.4166

\sigma_y^2 = \frac{13085}{5} - 50.6^2 = 2617 - 2560.36 = 56.64, \quad \sigma_y = 7.5260

r = \frac{18.16}{2.4166 \times 7.5260} = \frac{18.16}{18.187} \approx 0.998

A very strong positive linear correlation: hotter days went with more bottles sold.

Worked example

Worked example 2. The tomato seller's data from the previous lesson give $n = 5$ , $\sum x = 2000$ , $\sum y = 181$ , $\sum x^2 = 900\,000$ , $\sum y^2 = 7045$ , $\sum xy = 65\,400$ , with $x$ the price in Frw per kg and $y$ the mass sold in kg. Find $r$ .

$\bar{x} = 400$ , $\bar{y} = 36.2$ .

\operatorname{cov}(x, y) = \frac{65\,400}{5} - 400 \times 36.2 = 13\,080 - 14\,480 = -1400

\sigma_x = \sqrt{\frac{900\,000}{5} - 400^2} = \sqrt{20\,000} = 141.42, \qquad \sigma_y = \sqrt{\frac{7045}{5} - 36.2^2} = \sqrt{98.56} = 9.9277

r = \frac{-1400}{141.42 \times 9.9277} = \frac{-1400}{1404.0} \approx -0.997

A very strong negative linear correlation. Notice that the covariance, $- 1400$ , looks huge only because prices are in hundreds of francs; $r$ removes the effect of the units.

Properties of $r$

$-1 \le r \le 1$ always.
$r = 1$ or $r = - 1$ only when all the points lie exactly on a straight line (perfect correlation).
$r$ has the same sign as the covariance.
$r$ is symmetric: the correlation of $x$ with $y$ equals that of $y$ with $x$ .
$r$ does not change if either variable is rescaled by a positive factor or shifted by a constant: measuring the price in hundreds of francs, or the temperature in kelvin, leaves $r$ unchanged.
$r = 0$ means no linear correlation. There may still be a strong curved relationship.

Value of $∣ r ∣$	Common description (a rule of thumb, not a law)
$0.8 \le \|r\| \le 1$	strong
$0.5 \le \|r\| < 0.8$	moderate
$0 < ∣ r ∣ < 0.5$	weak
$r = 0$	no linear correlation

The Organic Chemistry Tutor works through r from the covariance and the two standard deviations. If a video divides by n − 1 instead of n, r is the same, because the divisor appears above and below the fraction and cancels.

Warning

Check your answer. If you ever get $∣ r ∣ > 1$ , you have made an arithmetic error, most often by using $\sum x^2$ where $(\sum x)^2$ was needed, or by forgetting to subtract $\bar{x}^2$ . Keep at least four significant figures in $\sigma_x$ and $\sigma_y$ until the last step, then round $r$ to 3 decimal places.

Tip

Calculator check: most scientific calculators have a two-variable statistics mode (often labelled "A+BX"). Enter the pairs and read off $r$ directly. Always show the table in an exam, then use the calculator to confirm.

✦

Practice

For the pairs $(1, 3)$ , $(2, 5)$ , $(3, 7)$ , find $\operatorname{cov}(x, y)$ .
Given $\operatorname{cov}(x, y) = -6$ , $\sigma_x = 2$ and $\sigma_y = 4$ , find $r$ and describe the correlation.
For $n = 6$ : $\sum x = 30$ , $\sum y = 48$ , $\sum xy = 270$ . Find $\operatorname{cov}(x, y)$ .
Find $r$ for $(1, 2)$ , $(2, 3)$ , $(3, 5)$ , $(4, 4)$ , $(5, 6)$ .
A student reports $r = 1.35$ . Without seeing the data, explain what you can say.
Heights are measured in metres and then again in centimetres. What happens to the covariance with mass, and to $r$ ? Explain.
For the points $(- 2, 4)$ , $(- 1, 1)$ , $(0, 0)$ , $(1, 1)$ , $(2, 4)$ , show that $r = 0$ . Does this mean $x$ and $y$ are unrelated?

Show answersHide answers

$\bar{x} = 2$ , $\bar{y} = 5$ , $\sum xy = 3 + 10 + 21 = 34$ . $\operatorname{cov} = \frac{34}{3} - 10 = \frac{4}{3} \approx 1.33$ .
$r = \frac{-6}{2 \times 4} = -0.75$ : moderate negative correlation.
$\bar{x} = 5$ , $\bar{y} = 8$ ; $\operatorname{cov} = \frac{270}{6} - 40 = 45 - 40 = 5$ .
$\bar{x} = 3$ , $\bar{y} = 4$ , $\sum xy = 69$ , $\sum x^2 = 55$ , $\sum y^2 = 90$ . $\operatorname{cov} = 13.8 - 12 = 1.8$ ; $\sigma_x^2 = 11 - 9 = 2$ ; $\sigma_y^2 = 18 - 16 = 2$ . $r = \frac{1.8}{\sqrt{2}\sqrt{2}} = 0.9$ .
It is wrong: $r$ can never be greater than 1, so there is a calculation error.
Every height is multiplied by 100, so the covariance is multiplied by 100, but $\sigma$ of height is also multiplied by 100, so $r$ is unchanged.
$\bar{x} = 0$ , $\sum xy = -8 - 1 + 0 + 1 + 8 = 0$ , so $\operatorname{cov} = 0$ and $r = 0$ . They are perfectly related by $y = x^{2}$ ; the relationship is just not linear.

Important

$\operatorname{cov}(x, y) = \frac{\sum xy}{n} - \bar{x}\bar{y}$ gives the direction; $r = \frac{\operatorname{cov}(x, y)}{\sigma_x \sigma_y}$ gives direction and strength on a scale from $- 1$ to $1$ .

Sources and credits

Text, data, diagram, worked examples and exercises: Written by Ganzaa. All data are illustrative.

Video: How To Calculate The Correlation Coefficient Using The Covariance Formula, The Organic Chemistry Tutor, embedded from YouTube.

1From a picture to a number

2Review: mean and standard deviation

3Why the covariance works

4Properties of r r r

From a picture to a number

Review: mean and standard deviation

Why the covariance works

Properties of $r$