Two-Variable Statistics

Lessons

Senior 6 · Basic Mathematics

Two-Variable Statistics

4 lessons

Lesson 3 of 4

Regression lines of y on x and x on y, and prediction

25 min

A line that predicts

The kiosk owner from the last lesson wants to plan her stock. The weather forecast says tomorrow will reach 26 °C: how many bottles should she expect to sell? Pearson's $r \approx 0.998$ says a straight line fits her data very well. This lesson finds that line and uses it.

Review: a line through a point with a given gradient

The line with gradient $m$ through the point $(x_{1}, y_{1})$ is

y - y_{1} = m (x - x_{1})

For example, the line with gradient 3 through $(2, 5)$ is $y - 5 = 3 (x - 2)$ , that is $y = 3 x - 1$ .

Definition

The regression line of $y$ on $x$ is the straight line that makes the sum of the squared vertical distances from the points to the line as small as possible (the least-squares line). It passes through the mean point $(\bar{x}, \bar{y})$ with gradient $b_{yx}$ :

y - \bar{y} = b_{yx}(x - \bar{x}), \qquad b_{yx} = \frac{\operatorname{cov}(x, y)}{\sigma_x^2}

The regression line of $x$ on $y$ minimises the squared horizontal distances:

x - \bar{x} = b_{xy}(y - \bar{y}), \qquad b_{xy} = \frac{\operatorname{cov}(x, y)}{\sigma_y^2}

$b_{yx}$ and $b_{xy}$ are the regression coefficients. Example: if $\operatorname{cov} = 6$ and $\sigma_x^2 = 4$ , then $b_{yx} = 1.5$ .

Which line to use

To predict $y$ from a known $x$ , use the line of $y$ on $x$ .
To predict $x$ from a known $y$ , use the line of $x$ on $y$ .

The two lines are different unless $r = \pm 1$ . They always cross at $(\bar{x}, \bar{y})$ . The closer $∣ r ∣$ is to 1, the smaller the angle between them; when $r = 0$ they are perpendicular, one horizontal ( $y = \bar{y}$ ) and one vertical ( $x = \bar{x}$ ).

Multiplying the two regression coefficients gives

b_{yx} \times b_{xy} = \frac{\operatorname{cov}(x, y)}{\sigma_x^2} \times \frac{\operatorname{cov}(x, y)}{\sigma_y^2} = \left(\frac{\operatorname{cov}(x, y)}{\sigma_x \sigma_y}\right)^2 = r^2

So $r = \pm\sqrt{b_{yx}\,b_{xy}}$ , with the sign of the regression coefficients (both are positive or both negative, like the covariance).

Worked example

Worked example 1. For the water data, $\bar{x} = 25.4$ , $\bar{y} = 50.6$ , $\operatorname{cov}(x, y) = 18.16$ , $\sigma_x^2 = 5.84$ . Find the line of $y$ on $x$ and predict the sales at 26 °C.

b_{yx} = \frac{18.16}{5.84} = 3.1096

y - 50.6 = 3.1096(x - 25.4) \;\Rightarrow\; y = 3.1096x - 78.98 + 50.6 = 3.110x - 28.38

At $x = 26$ : $y = 3.1096 \times 26 - 28.38 = 80.85 - 28.38 = 52.47$ . She should expect about 52 bottles.

Worked example

Worked example 2. On one day she sold 60 bottles but lost the temperature record. Estimate it. ( $\sigma_y^2 = 56.64$ .)

We know $y$ and want $x$ , so we need the line of $x$ on $y$ :

b_{xy} = \frac{18.16}{56.64} = 0.3206

x - 25.4 = 0.3206(y - 50.6) \;\Rightarrow\; x = 0.3206y + 9.18

At $y = 60$ : $x = 0.3206 \times 60 + 9.18 = 19.24 + 9.18 = 28.4$ . The temperature was about 28.4 °C.

Check: $b_{yx} \times b_{xy} = 3.1096 \times 0.3206 = 0.997$ , and $r^{2} = {0.9985}^{2} = 0.997$ . ✓

Worked example

Worked example 3. The two regression lines of a data set are $y = 0.8 x + 4$ ( $y$ on $x$ ) and $x = 0.45 y + 3$ ( $x$ on $y$ ). Find $\bar{x}$ , $\bar{y}$ and $r$ .

Both lines pass through $(\bar{x}, \bar{y})$ , so solve them together. Substitute the first into the second:

x = 0.45(0.8x + 4) + 3 = 0.36x + 4.8 \;\Rightarrow\; 0.64x = 4.8 \;\Rightarrow\; x = 7.5

Then $y = 0.8 \times 7.5 + 4 = 10$ . So $\bar{x} = 7.5$ and $\bar{y} = 10$ .

r^2 = 0.8 \times 0.45 = 0.36 \;\Rightarrow\; r = 0.6

$r$ is positive because both coefficients are positive.

Khan Academy finds the least-squares line of y on x from the means, the standard deviations and r. Its slope formula, r times the ratio of the standard deviations, equals cov(x, y) divided by the variance of x used in this lesson.

Warning

Do not rearrange one line to get the other. Making $x$ the subject of $y = 3.110 x - 28.38$ gives $x = 0.3216 y + 9.13$ , which is not the line of $x$ on $y$ ( $x = 0.3206 y + 9.18$ ). The two lines minimise different distances. Use $y$ on $x$ to find $y$ , and $x$ on $y$ to find $x$ .

Tip

The variable you are predicting is the one that comes first: " $y$ on $x$ " predicts $y$ . And the regression coefficient for it divides by the variance of the other variable.

✦

Practice

Find the equation of the line with gradient $- 2$ through $(3, 7)$ .
Given $\bar{x} = 4$ , $\bar{y} = 10$ , $\operatorname{cov}(x, y) = 9$ , $\sigma_x^2 = 6$ : find the regression line of $y$ on $x$ and estimate $y$ when $x = 6$ .
With the same data and $\sigma_y^2 = 25$ , find the line of $x$ on $y$ and estimate $x$ when $y = 15$ .
For the revision data of lesson 1, $\bar{x} = 5.5$ , $\bar{y} = 54.625$ , $\operatorname{cov} = 30.5625$ , $\sigma_x^2 = 5.25$ . Predict the mark of a student who revises 6.5 hours.
Regression lines are $y = - 1.2 x + 20$ and $x = - 0.5 y + 11$ . Find the mean point and $r$ .
Explain why the two regression lines coincide when $r = 1$ .
A student finds $b_{yx} = 1.6$ and $b_{xy} = 0.9$ . Show that this is impossible.

Show answersHide answers

$y - 7 = - 2 (x - 3)$ , so $y = - 2 x + 13$ .
$b_{yx} = 1.5$ ; $y - 10 = 1.5 (x - 4)$ , so $y = 1.5 x + 4$ . At $x = 6$ , $y = 13$ .
$b_{xy} = \frac{9}{25} = 0.36$ ; $x - 4 = 0.36 (y - 10)$ , so $x = 0.36 y + 0.4$ . At $y = 15$ , $x = 5.8$ .
$b_{yx} = \frac{30.5625}{5.25} = 5.821$ ; $y = 5.821 x + 22.61$ . At $x = 6.5$ , $y \approx 60.5$ , so about 60%.
$x = - 0.5 (- 1.2 x + 20) + 11 = 0.6 x + 1$ , so $0.4 x = 1$ , $x = 2.5$ , $y = 17$ . $r^2 = 1.2 \times 0.5 = 0.6$ , $r = -\sqrt{0.6} \approx -0.775$ (negative, like the coefficients).
If $r = 1$ every point lies on one straight line; both the vertical and horizontal distances are then zero for that line, so both methods choose it.
$r^2 = 1.6 \times 0.9 = 1.44$ gives $∣ r ∣ = 1.2 > 1$ , which cannot happen.

Important

Both regression lines pass through $(\bar{x}, \bar{y})$ . Use $y$ on $x$ to predict $y$ , $x$ on $y$ to predict $x$ . And $b_{yx} \, b_{xy} = r^2$ .

Sources and credits

Text, data, diagram, worked examples and exercises: Written by Ganzaa. All data are illustrative.

Video: Calculating the equation of a regression line, Khan Academy, embedded from YouTube.

1A line that predicts

2Review: a line through a point with a given gradient

3Which line to use

A line that predicts

Review: a line through a point with a given gradient

Which line to use