Transformation of Matrices

Lessons

Senior 6 · Mathematics

Transformation of Matrices

5 lessons

Lesson 4 of 5

Eigenvalues and eigenvectors

35 min

Directions a matrix does not turn

Most vectors change direction when a matrix acts on them. In the figure below, $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ sends $(1, 0)$ to $(4, 2)$ , a different direction. But $(1, 1)$ goes to $(5, 5)$ , which is the same arrow stretched 5 times, and $(1, - 2)$ goes to $(2, - 4)$ , the same arrow stretched twice. These special directions are the "own" directions of the matrix: in German, eigen means "own". They explain everything a matrix does when it is applied again and again, which is the subject of lesson 5. Engineers use them to find the natural frequencies of a bridge, and search engines to rank web pages.

Definition

Let $A$ be a square matrix of order $n$ . A real number $\lambda$ is an eigenvalue of $A$ if there is a non-zero vector $\mathbf{v}$ with

A\mathbf{v} = \lambda\mathbf{v} .

Such a $\mathbf{v}$ is an eigenvector of $A$ for $\lambda$ . Example: $\begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}\begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 5 \\ 5 \end{pmatrix} = 5\begin{pmatrix} 1 \\ 1 \end{pmatrix}$ , so 5 is an eigenvalue with eigenvector $(1, 1)$ .

The characteristic polynomial of $A$ is $p(\lambda) = \det(A - \lambda I)$ , and $p(\lambda) = 0$ is the characteristic equation. For an eigenvalue $\lambda$ , the eigenspace $E_\lambda = \operatorname{Ker}(A - \lambda I)$ is the set of its eigenvectors together with $\mathbf{0}$ .

Under

A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}

, vectors on the two dashed lines stay on their lines and are only stretched; every other vector is turned.

Important

Theorem. $\lambda$ is an eigenvalue of $A$ if and only if $\det(A - \lambda I) = 0$ .

Reason. $A\mathbf{v} = \lambda\mathbf{v}$ is the same as $(A - \lambda I)\mathbf{v} = \mathbf{0}$ . This square homogeneous system has a non-zero solution exactly when its determinant is 0 (lesson 2).

For a $2 \times 2$ matrix the characteristic polynomial is $\lambda^2 - (\operatorname{tr} A)\lambda + \det A$ , where the trace $\operatorname{tr} A$ is the sum of the diagonal entries.

The method

Expand $\det(A - \lambda I)$ and solve $p(\lambda) = 0$ : the roots are the eigenvalues.
For each eigenvalue $\lambda$ , solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ by row reduction. Its solutions, except $\mathbf{0}$ , are the eigenvectors; a basis of the solutions is a basis of $E_\lambda$ .
Check: $A\mathbf{v}$ must equal $\lambda\mathbf{v}$ .

Worked example 1: a $2 \times 2$ matrix

Find the eigenvalues and eigenvectors of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ .

Solution. $\operatorname{tr} A = 7$ and $\det A = 12 - 2 = 10$ , so

p(\lambda) = \lambda^2 - 7\lambda + 10 = (\lambda - 2)(\lambda - 5) , \qquad \lambda_1 = 5 ,\ \lambda_2 = 2 .

$\lambda = 5$ : $A - 5I = \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \sim \begin{pmatrix} 1 & -1 \\ 0 & 0 \end{pmatrix}$ , so $x = y$ : $E_5 = \operatorname{span}\{(1, 1)\}$ .

$\lambda = 2$ : $A - 2I = \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} \sim \begin{pmatrix} 2 & 1 \\ 0 & 0 \end{pmatrix}$ , so $y = - 2 x$ : $E_2 = \operatorname{span}\{(1, -2)\}$ .

Check: $A(1, -2) = (4 - 2,\ 2 - 6) = (2, -4) = 2(1, -2)$ . The eigenvectors are all the non-zero multiples, such as $(3, 3)$ or $(- 2, 4)$ .

Worked example 2: a $3 \times 3$ matrix with a repeated eigenvalue

Find the eigenvalues and eigenspaces of $S = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{pmatrix}$ .

Solution. Expand along row 1:

\det(S - \lambda I) = (2 - \lambda)\big[(2 - \lambda)^2 - 1\big] - \big[(2 - \lambda) - 1\big] + \big[1 - (2 - \lambda)\big] .

Now $(2 - \lambda)^2 - 1 = (1 - \lambda)(3 - \lambda)$ , and the last two terms are $-(1 - \lambda) + (\lambda - 1) = 2(\lambda - 1)$ . Taking out the factor $(\lambda - 1)$ :

\det(S - \lambda I) = (\lambda - 1)\big[(2 - \lambda)(\lambda - 3) + 2\big] = (\lambda - 1)(-\lambda^2 + 5\lambda - 4) = -(\lambda - 1)^2(\lambda - 4) .

The eigenvalues are $4$ and $1$ (a double root).

$\lambda = 4$ : $S - 4I = \begin{pmatrix} -2 & 1 & 1 \\ 1 & -2 & 1 \\ 1 & 1 & -2 \end{pmatrix} \sim \begin{pmatrix} 1 & 0 & -1 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{pmatrix}$ , so $x = y = z$ : $E_4 = \operatorname{span}\{(1, 1, 1)\}$ .

$\lambda = 1$ : $S - I = \begin{pmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{pmatrix}$ , rank 1, the single equation $x + y + z = 0$ with two free variables: $E_1 = \operatorname{span}\{(-1, 1, 0), (-1, 0, 1)\}$ , a plane.

Check with the trace and determinant: $4 + 1 + 1 = 6 = \operatorname{tr} S$ and $4 \times 1 \times 1 = 4 = \det S$ .

Important

Useful facts. For a square matrix $A$ of order $n$ whose characteristic polynomial has $n$ real roots $\lambda_1, \dots, \lambda_n$ (counted with repetition):

$\lambda_1 + \lambda_2 + \cdots + \lambda_n = \operatorname{tr} A$ and $\lambda_1\lambda_2\cdots\lambda_n = \det A$ ;
the eigenvalues of a triangular (or diagonal) matrix are its diagonal entries;
if $A\mathbf{v} = \lambda\mathbf{v}$ , then $A^k\mathbf{v} = \lambda^k\mathbf{v}$ for every positive integer $k$ , and, if $A$ is invertible, $A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}$ .

Reason for the last: $A^2\mathbf{v} = A(\lambda\mathbf{v}) = \lambda A\mathbf{v} = \lambda^2\mathbf{v}$ , and so on. And $\lambda \ne 0$ when $A$ is invertible, since $\det A$ is the product of the eigenvalues; then $\mathbf{v} = A^{-1}(\lambda\mathbf{v})$ gives $A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}$ .

Worked example 3: using the facts without a determinant

(a) Write down the eigenvalues of $T = \begin{pmatrix} 3 & 5 & -1 \\ 0 & -2 & 4 \\ 0 & 0 & 7 \end{pmatrix}$ . (b) Hence find the eigenvalues of $T^{2}$ and of $T^{- 1}$ , and $\det T$ .

Solution. (a) $T$ is upper triangular, so $\det(T - \lambda I) = (3 - \lambda)(-2 - \lambda)(7 - \lambda)$ : the eigenvalues are $3, - 2, 7$ . (b) $T^{2}$ has eigenvalues $9, 4, 49$ , and $T^{- 1}$ has $\frac{1}{3}, -\frac{1}{2}, \frac{1}{7}$ . $\det T = 3 \times (-2) \times 7 = -42$ .

Warning

An eigenvector is never the zero vector, although $\lambda = 0$ is allowed as an eigenvalue (it happens exactly when $\det A = 0$ ).

Some real matrices have no real eigenvalues. The rotation through $90^\circ$ , $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ , has $p(\lambda) = \lambda^2 + 1$ , with no real root: a quarter turn leaves no direction unchanged. Say so rather than inventing eigenvectors.

Subtract $\lambda$ from the diagonal only. $A - \lambda I$ changes the diagonal entries, not every entry.

Tip

After finding $p(\lambda)$ , check it at once: the sum of your roots must be the trace, and their product the determinant. For a $3 \times 3$ matrix, the coefficient of $\lambda^2$ in $-p(\lambda)$ gives the same check: $-p(\lambda) = \lambda^3 - (\operatorname{tr} A)\lambda^2 + \cdots - \det A$ .

3Blue1Brown: Eigenvectors and eigenvalues, Chapter 14 of Essence of Linear Algebra (about 17 min). Watch the vectors that stay on their own span while the plane is transformed, and why a rotation has none, as in the warning above.

✦

Practice

Questions 1 to 4 are routine; 5 and 6 need more care; 7 asks for a proof.

Find the eigenvalues and an eigenvector for each eigenvalue of $\begin{pmatrix} 1 & 2 \\ 3 & 2 \end{pmatrix}$ .
Show that $\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ has no real eigenvalues.
Show that $(1, - 1, 0)$ is an eigenvector of $B = \begin{pmatrix} 1 & 2 & 0 \\ 2 & 1 & 0 \\ 0 & 0 & 3 \end{pmatrix}$ , and give its eigenvalue.
Write down the eigenvalues of $\begin{pmatrix} 2 & 7 & -3 \\ 0 & -1 & 5 \\ 0 & 0 & 4 \end{pmatrix}$ .
Find all the eigenvalues of $B$ in question 3, and a basis of each eigenspace.
A $2 \times 2$ matrix $A$ has trace 6 and determinant 8. Find the eigenvalues of $A$ , of $A^{2}$ and of $A^{- 1}$ .
Prove that a square matrix $A$ and its transpose $A^{T}$ have the same eigenvalues. (Use $\det M^T = \det M$ .)

Show answersHide answers

$p(\lambda) = \lambda^2 - 3\lambda - 4 = (\lambda - 4)(\lambda + 1)$ : $\lambda = 4$ with $(2, 3)$ ; $\lambda = -1$ with $(1, - 1)$ .
$p(\lambda) = \lambda^2 + 1$ , whose discriminant is $-4 \lt 0$ .
$B(1, -1, 0) = (-1, 1, 0) = -1 \cdot (1, -1, 0)$ : eigenvalue $- 1$ .
$2, - 1, 4$
$p(\lambda) = (3 - \lambda)\big[(1 - \lambda)^2 - 4\big] = -(\lambda - 3)^2(\lambda + 1)$ . $E_{-1} = \operatorname{span}\{(1, -1, 0)\}$ ; $E_3 = \operatorname{span}\{(1, 1, 0), (0, 0, 1)\}$ .
$\lambda^2 - 6\lambda + 8 = 0$ : $2$ and $4$ ; $A^{2}$ : $4$ and $16$ ; $A^{- 1}$ : $\frac{1}{2}$ and $\frac{1}{4}$ .
$(A - \lambda I)^T = A^T - \lambda I$ , so $\det(A^T - \lambda I) = \det(A - \lambda I)$ : the same characteristic polynomial.

Sources and credits

Written by Ganzaa: the explanations, the proofs, the diagram (drawn from the computed images of the three vectors), the worked examples, the practice items and the quiz questions. Every eigenvector was checked by computing $A\mathbf{v}$ .
Video: Eigenvectors and eigenvalues, Chapter 14 of Essence of Linear Algebra, 3Blue1Brown (Grant Sanderson), embedded from YouTube.

1Directions a matrix does not turn

2The method

Directions a matrix does not turn

The method