generalized eigenvector#Overview and definition

There are several equivalent ways to define an ordinary eigenvector.{{harvtxt|Anton|1987|pp=301–302}}{{harvtxt|Beauregard|Fraleigh|1973|p=266}}{{harvtxt|Burden|Faires|1993|p=401}}{{harvtxt|Golub|Van Loan|1996|pp=310–311}}{{harvtxt|Harper|1976|p=58}}{{harvtxt|Herstein|1964|p=225}}{{harvtxt|Kreyszig|1972|pp=273,684}}{{harvtxt|Nering|1970|p=104}} For our purposes, an eigenvector $\backslash mathbf\; u$ associated with an eigenvalue $\backslash lambda$ of an $n$ × $n$ matrix $A$ is a nonzero vector for which $(A\; -\; \backslash lambda\; I)\; \backslash mathbf\; u\; =\; \backslash mathbf\; 0$, where $I$ is the $n$ × $n$ identity matrix and $\backslash mathbf\; 0$ is the zero vector of length $n$.{{harvtxt|Burden|Faires|1993|p=401}} That is, $\backslash mathbf\; u$ is in the kernel of the transformation $(A\; -\; \backslash lambda\; I)$. If $A$ has $n$ linearly independent eigenvectors, then $A$ is similar to a diagonal matrix $D$. That is, there exists an invertible matrix $M$ such that $A$ is diagonalizable through the similarity transformation $D\; =\; M^\{-1\}AM$.{{harvtxt|Beauregard|Fraleigh|1973|pp=270–274}}{{harvtxt|Bronson|1970|pp=179–183}} The matrix $D$ is called a spectral matrix for $A$. The matrix $M$ is called a modal matrix for $A$.{{harvtxt|Bronson|1970|p=181}} Diagonalizable matrices are of particular interest since matrix functions of them can be computed easily.{{harvtxt|Bronson|1970|p=179}}

On the other hand, if $A$ does not have $n$ linearly independent eigenvectors associated with it, then $A$ is not diagonalizable.{{harvtxt|Beauregard|Fraleigh|1973|pp=270–274}}{{harvtxt|Bronson|1970|pp=179–183}}

Definition: A vector $\backslash mathbf\; x\_m$ is a generalized eigenvector of rank m of the matrix $A$ and corresponding to the eigenvalue $\backslash lambda$ if

:$(A\; -\; \backslash lambda\; I)^m\; \backslash mathbf\; x\_m\; =\; \backslash mathbf\; 0$

but

:$(A\; -\; \backslash lambda\; I)^\{m-1\}\; \backslash mathbf\; x\_m\; \backslash ne\; \backslash mathbf\; 0.$ {{harvtxt|Bronson|1970|p=189}}

Clearly, a generalized eigenvector of rank 1 is an ordinary eigenvector.{{harvtxt|Bronson|1970|pp=190,202}} Every $n$ × $n$ matrix $A$ has $n$ linearly independent generalized eigenvectors associated with it and can be shown to be similar to an "almost diagonal" matrix $J$ in Jordan normal form.{{harvtxt|Bronson|1970|pp=189,203}} That is, there exists an invertible matrix $M$ such that $J\; =\; M^\{-1\}AM$.{{harvtxt|Bronson|1970|pp=206–207}} The matrix $M$ in this case is called a generalized modal matrix for $A$.{{harvtxt|Bronson|1970|p=205}} If $\backslash lambda$ is an eigenvalue of algebraic multiplicity $\backslash mu$, then $A$ will have $\backslash mu$ linearly independent generalized eigenvectors corresponding to $\backslash lambda$.{{harvtxt|Bronson|1970|p=196}} These results, in turn, provide a straightforward method for computing certain matrix functions of $A$.{{harvtxt|Bronson|1970|pp=189,209–215}}

{{anchor|Note}}Note: For an $n\; \backslash times\; n$ matrix $A$ over a field $F$ to be expressed in Jordan normal form, all eigenvalues of $A$ must be in $F$. That is, the characteristic polynomial $f(x)$ must factor completely into linear factors; $F$ must be an algebraically closed field. For example, if $A$ has real-valued elements, then it may be necessary for the eigenvalues and the components of the eigenvectors to have complex values.{{harvtxt|Golub|Van Loan|1996|p=316}}{{harvtxt|Herstein|1964|p=259}}{{harvtxt|Nering|1970|p=118}}

The set spanned by all generalized eigenvectors for a given $\backslash lambda$ forms the generalized eigenspace for $\backslash lambda$.{{harvtxt|Nering|1970|p=118}}

Here are some examples to illustrate the concept of generalized eigenvectors. Some of the details will be described later.

This example is simple but clearly illustrates the point. This type of matrix is used frequently in textbooks.{{harvtxt|Nering|1970|p=118}}{{harvtxt|Herstein|1964|p=261}}{{harvtxt|Beauregard|Fraleigh|1973|p=310}}

Suppose

:

Then there is only one eigenvalue, $\backslash lambda\; =\; 1$, and its algebraic multiplicity is $m=2$.

Notice that this matrix is in Jordan normal form but is not diagonal. Hence, this matrix is not diagonalizable. Since there is one superdiagonal entry, there will be one generalized eigenvector of rank greater than 1 (or one could note that the vector space $V$ is of dimension 2, so there can be at most one generalized eigenvector of rank greater than 1). Alternatively, one could compute the dimension of the nullspace of $A\; -\; \backslash lambda\; I$ to be $p=1$, and thus there are $m-p=1$ generalized eigenvectors of rank greater than 1.

The ordinary eigenvector $\backslash mathbf\; v\_1=\backslash begin\{pmatrix\}1\; \backslash \backslash 0\; \backslash end\{pmatrix\}$ is computed as usual (see the eigenvector page for examples). Using this eigenvector, we compute the generalized eigenvector $\backslash mathbf\; v\_2$ by solving

:$(A-\backslash lambda\; I)\; \backslash mathbf\; v\_2\; =\; \backslash mathbf\; v\_1.$

Writing out the values:

:

\begin{pmatrix}1 \\0 \end{pmatrix}.

This simplifies to

:$v\_\{22\}=\; 1.$

The element $v\_\{21\}$ has no restrictions. The generalized eigenvector of rank 2 is then $\backslash mathbf\; v\_2=\backslash begin\{pmatrix\}a\; \backslash \backslash 1\; \backslash end\{pmatrix\}$, where a can have any scalar value. The choice of a = 0 is usually the simplest.

Note that

:

\begin{pmatrix}1 \\0 \end{pmatrix} = \mathbf v_1,

so that $\backslash mathbf\; v\_2$ is a generalized eigenvector, because

:

\begin{pmatrix}0 \\0 \end{pmatrix} = \mathbf 0,

so that $\backslash mathbf\; v\_1$ is an ordinary eigenvector, and that $\backslash mathbf\; v\_1$ and $\backslash mathbf\; v\_2$ are linearly independent and hence constitute a basis for the vector space $V$.

This example is more complex than Example 1. Unfortunately, it is a little difficult to construct an interesting example of low order.{{harvtxt|Nering|1970|pp=122,123}}

The matrix

:$A\; =\; \backslash begin\{pmatrix\}$

1 & 0 & 0 & 0 & 0 \\

3 & 1 & 0 & 0 & 0 \\

6 & 3 & 2 & 0 & 0 \\

10 & 6 & 3 & 2 & 0 \\

15 & 10 & 6 & 3 & 2

\end{pmatrix}

has eigenvalues $\backslash lambda\_1\; =\; 1$ and $\backslash lambda\_2\; =\; 2$ with algebraic multiplicities $\backslash mu\_1\; =\; 2$ and $\backslash mu\_2\; =\; 3$, but geometric multiplicities $\backslash gamma\_1\; =\; 1$ and $\backslash gamma\_2\; =\; 1$.

The generalized eigenspaces of $A$ are calculated below.

$\backslash mathbf\; x\_1$ is the ordinary eigenvector associated with $\backslash lambda\_1$.

$\backslash mathbf\; x\_2$ is a generalized eigenvector associated with $\backslash lambda\_1$.

$\backslash mathbf\; y\_1$ is the ordinary eigenvector associated with $\backslash lambda\_2$.

$\backslash mathbf\; y\_2$ and $\backslash mathbf\; y\_3$ are generalized eigenvectors associated with $\backslash lambda\_2$.

:$(A-1\; I)\; \backslash mathbf\; x\_1$

= \begin{pmatrix}

0 & 0 & 0 & 0 & 0 \\

3 & 0 & 0 & 0 & 0 \\

6 & 3 & 1 & 0 & 0 \\

10 & 6 & 3 & 1 & 0 \\

15 & 10 & 6 & 3 & 1

\end{pmatrix}\begin{pmatrix}

0 \\ 3 \\ -9 \\ 9 \\ -3

\end{pmatrix} = \begin{pmatrix}

0 \\ 0 \\ 0 \\ 0 \\ 0

\end{pmatrix} = \mathbf 0 ,

:$(A\; -\; 1\; I)\; \backslash mathbf\; x\_2$

= \begin{pmatrix}

0 & 0 & 0 & 0 & 0 \\

3 & 0 & 0 & 0 & 0 \\

6 & 3 & 1 & 0 & 0 \\

10 & 6 & 3 & 1 & 0 \\

15 & 10 & 6 & 3 & 1

\end{pmatrix} \begin{pmatrix}

1 \\ -15 \\ 30 \\ -1 \\ -45

\end{pmatrix} = \begin{pmatrix}

0 \\ 3 \\ -9 \\ 9 \\ -3

\end{pmatrix} = \mathbf x_1 ,

:$(A\; -\; 2\; I)\; \backslash mathbf\; y\_1$

= \begin{pmatrix}

-1 & 0 & 0 & 0 & 0 \\

3 & -1 & 0 & 0 & 0 \\

6 & 3 & 0 & 0 & 0 \\

10 & 6 & 3 & 0 & 0 \\

15 & 10 & 6 & 3 & 0

\end{pmatrix} \begin{pmatrix}

0 \\ 0 \\ 0 \\ 0 \\ 9

\end{pmatrix} = \begin{pmatrix}

0 \\ 0 \\ 0 \\ 0 \\ 0

\end{pmatrix} = \mathbf 0 ,

:$(A\; -\; 2\; I)\; \backslash mathbf\; y\_2\; =\; \backslash begin\{pmatrix\}$

-1 & 0 & 0 & 0 & 0 \\

3 & -1 & 0 & 0 & 0 \\

6 & 3 & 0 & 0 & 0 \\

10 & 6 & 3 & 0 & 0 \\

15 & 10 & 6 & 3 & 0

\end{pmatrix} \begin{pmatrix}

0 \\ 0 \\ 0 \\ 3 \\ 0

\end{pmatrix} = \begin{pmatrix}

0 \\ 0 \\ 0 \\ 0 \\ 9

\end{pmatrix} = \mathbf y_1 ,

:$(A\; -\; 2\; I)\; \backslash mathbf\; y\_3\; =\; \backslash begin\{pmatrix\}$

-1 & 0 & 0 & 0 & 0 \\

3 & -1 & 0 & 0 & 0 \\

6 & 3 & 0 & 0 & 0 \\

10 & 6 & 3 & 0 & 0 \\

15 & 10 & 6 & 3 & 0

\end{pmatrix} \begin{pmatrix}

0 \\ 0 \\ 1 \\ -2 \\ 0

\end{pmatrix} = \begin{pmatrix}

0 \\ 0 \\ 0 \\ 3 \\ 0

\end{pmatrix} = \mathbf y_2 .

This results in a basis for each of the generalized eigenspaces of $A$.

Together the two chains of generalized eigenvectors span the space of all 5-dimensional column vectors.

:$$

\left\{ \mathbf x_1, \mathbf x_2 \right\} =

\left\{

\begin{pmatrix} 0 \\ 3 \\ -9 \\ 9 \\ -3 \end{pmatrix},

\begin{pmatrix} 1 \\ -15 \\ 30 \\ -1 \\ -45 \end{pmatrix}

\right\},

\left\{ \mathbf y_1, \mathbf y_2, \mathbf y_3 \right\} =

\left\{

\begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 9 \end{pmatrix},

\begin{pmatrix} 0 \\ 0 \\ 0 \\ 3 \\ 0 \end{pmatrix},

\begin{pmatrix} 0 \\ 0 \\ 1 \\ -2 \\ 0 \end{pmatrix}

\right\}.

An "almost diagonal" matrix $J$ in Jordan normal form, similar to $A$ is obtained as follows:

:$$

M =

\begin{pmatrix} \mathbf x_1 & \mathbf x_2 & \mathbf y_1 & \mathbf y_2 & \mathbf y_3 \end{pmatrix} =

\begin{pmatrix}

0 & 1 & 0 &0& 0 \\

3 & -15 & 0 &0& 0 \\

-9 & 30 & 0 &0& 1 \\

9 & -1 & 0 &3& -2 \\

-3 & -45 & 9 &0& 0

\end{pmatrix},

:$J\; =\; \backslash begin\{pmatrix\}$

1 & 1 & 0 & 0 & 0 \\

0 & 1 & 0 & 0 & 0 \\

0 & 0 & 2 & 1 & 0 \\

0 & 0 & 0 & 2 & 1 \\

0 & 0 & 0 & 0 & 2

\end{pmatrix},

where $M$ is a generalized modal matrix for $A$, the columns of $M$ are a canonical basis for $A$, and $AM\; =\; MJ$.{{harvtxt|Bronson|1970|pp=189–209}}

Definition: Let $\backslash mathbf\; x\_m$ be a generalized eigenvector of rank m corresponding to the matrix $A$ and the eigenvalue $\backslash lambda$. The chain generated by $\backslash mathbf\; x\_m$ is a set of vectors $\backslash left\backslash \{\; \backslash mathbf\; x\_m,\; \backslash mathbf\; x\_\{m-1\},\; \backslash dots\; ,\; \backslash mathbf\; x\_1\; \backslash right\backslash \}$ given by

{{NumBlk|::|

$\backslash mathbf\; x\_\{m-1\}\; =\; (A\; -\; \backslash lambda\; I)\; \backslash mathbf\; x\_m,$

$\backslash mathbf\; x\_\{m-2\}\; =\; (A\; -\; \backslash lambda\; I)^2\; \backslash mathbf\; x\_m\; =\; (A\; -\; \backslash lambda\; I)\; \backslash mathbf\; x\_\{m-1\},$

$\backslash mathbf\; x\_\{m-3\}\; =\; (A\; -\; \backslash lambda\; I)^3\; \backslash mathbf\; x\_m\; =\; (A\; -\; \backslash lambda\; I)\; \backslash mathbf\; x\_\{m-2\},$

::$\backslash vdots$

$\backslash mathbf\; x\_1\; =\; (A\; -\; \backslash lambda\; I)^\{m-1\}\; \backslash mathbf\; x\_m\; =\; (A\; -\; \backslash lambda\; I)\; \backslash mathbf\; x\_2.$

|{{EquationRef|1}}}}

where $\backslash mathbf\; x\_1$ is always an ordinary eigenvector with a given eigenvalue $\backslash lambda$. Thus, in general,

{{NumBlk|::|$\backslash mathbf\; x\_j\; =\; (A\; -\; \backslash lambda\; I)^\{m-j\}\; \backslash mathbf\; x\_m\; =\; (A\; -\; \backslash lambda\; I)\; \backslash mathbf\; x\_\{j+1\}\; \backslash qquad\; (j\; =\; 1,\; 2,\; \backslash dots\; ,\; m\; -\; 1).$|{{EquationRef|2}}}}

The vector $\backslash mathbf\; x\_j$, given by ({{EquationNote|2}}), is a generalized eigenvector of rank j corresponding to the eigenvalue $\backslash lambda$. A chain is a linearly independent set of vectors.{{harvtxt|Bronson|1970|pp=194–195}}

{{Main|Canonical basis#Linear algebra}}

Definition: A set of n linearly independent generalized eigenvectors is a canonical basis if it is composed entirely of Jordan chains.

Thus, once we have determined that a generalized eigenvector of rank m is in a canonical basis, it follows that the m − 1 vectors $\backslash mathbf\; x\_\{m-1\},\; \backslash mathbf\; x\_\{m-2\},\; \backslash ldots\; ,\; \backslash mathbf\; x\_1$ that are in the Jordan chain generated by $\backslash mathbf\; x\_m$ are also in the canonical basis.{{harvtxt|Bronson|1970|pp=196,197}}

Let $\backslash lambda\_i$ be an eigenvalue of $A$ of algebraic multiplicity $\backslash mu\_i$. First, find the ranks (matrix ranks) of the matrices $(A\; -\; \backslash lambda\_i\; I),\; (A\; -\; \backslash lambda\_i\; I)^2,\; \backslash ldots\; ,\; (A\; -\; \backslash lambda\_i\; I)^\{m\_i\}$. The integer $m\_i$ is determined to be the first integer for which $(A\; -\; \backslash lambda\_i\; I)^\{m\_i\}$ has rank $n\; -\; \backslash mu\_i$ (n being the number of rows or columns of $A$, that is, $A$ is n × n).

Now define

:$\backslash rho\_k\; =\; \backslash operatorname\{rank\}(A\; -\; \backslash lambda\_i\; I)^\{k-1\}\; -\; \backslash operatorname\{rank\}(A\; -\; \backslash lambda\_i\; I)^k\; \backslash qquad\; (k\; =\; 1,\; 2,\; \backslash ldots\; ,\; m\_i).$

The variable $\backslash rho\_k$ designates the number of linearly independent generalized eigenvectors of rank k corresponding to the eigenvalue $\backslash lambda\_i$ that will appear in a canonical basis for $A$. Note that

:$\backslash operatorname\{rank\}(A\; -\; \backslash lambda\_i\; I)^0\; =\; \backslash operatorname\{rank\}(I)\; =\; n$.{{harvtxt|Bronson|1970|pp=197,198}}

In the preceding sections we have seen techniques for obtaining the $n$ linearly independent generalized eigenvectors of a canonical basis for the vector space $V$ associated with an $n\; \backslash times\; n$ matrix $A$. These techniques can be combined into a procedure:

:Solve the characteristic equation of $A$ for eigenvalues $\backslash lambda\_i$ and their algebraic multiplicities $\backslash mu\_i$;

:For each $\backslash lambda\_i\; :$

::Determine $n\; -\; \backslash mu\_i$;

::Determine $m\_i$;

::Determine $\backslash rho\_k$ for $(k\; =\; 1,\; \backslash ldots\; ,\; m\_i)$;

::Determine each Jordan chain for $\backslash lambda\_i$;

The matrix

:$$

A =

\begin{pmatrix}

5 & 1 & -2 & 4 \\

0 & 5 & 2 & 2 \\

0 & 0 & 5 & 3 \\

0 & 0 & 0 & 4

\end{pmatrix}

has an eigenvalue $\backslash lambda\_1\; =\; 5$ of algebraic multiplicity $\backslash mu\_1\; =\; 3$ and an eigenvalue $\backslash lambda\_2\; =\; 4$ of algebraic multiplicity $\backslash mu\_2\; =\; 1$. We also have $n=4$. For $\backslash lambda\_1$ we have $n\; -\; \backslash mu\_1\; =\; 4\; -\; 3\; =\; 1$.

:$$

(A - 5I) =

\begin{pmatrix}

0 & 1 & -2 & 4 \\

0 & 0 & 2 & 2 \\

0 & 0 & 0 & 3 \\

0 & 0 & 0 & -1

\end{pmatrix},

\qquad \operatorname{rank}(A - 5I) = 3.

:$$

(A - 5I)^2 =

\begin{pmatrix}

0 & 0 & 2 & -8 \\

0 & 0 & 0 & 4 \\

0 & 0 & 0 & -3 \\

0 & 0 & 0 & 1

\end{pmatrix},

\qquad \operatorname{rank}(A - 5I)^2 = 2.

:$$

(A - 5I)^3 =

\begin{pmatrix}

0 & 0 & 0 & 14 \\

0 & 0 & 0 & -4 \\

0 & 0 & 0 & 3 \\

0 & 0 & 0 & -1

\end{pmatrix},

\qquad \operatorname{rank}(A - 5I)^3 = 1.

The first integer $m\_1$ for which $(A\; -\; 5I)^\{m\_1\}$ has rank $n\; -\; \backslash mu\_1\; =\; 1$ is $m\_1\; =\; 3$.

We now define

:$\backslash rho\_3\; =\; \backslash operatorname\{rank\}(A\; -\; 5I)^2\; -\; \backslash operatorname\{rank\}(A\; -\; 5I)^3\; =\; 2\; -\; 1\; =\; 1\; ,$

:$\backslash rho\_2\; =\; \backslash operatorname\{rank\}(A\; -\; 5I)^1\; -\; \backslash operatorname\{rank\}(A\; -\; 5I)^2\; =\; 3\; -\; 2\; =\; 1\; ,$

:$\backslash rho\_1\; =\; \backslash operatorname\{rank\}(A\; -\; 5I)^0\; -\; \backslash operatorname\{rank\}(A\; -\; 5I)^1\; =\; 4\; -\; 3\; =\; 1\; .$

Consequently, there will be three linearly independent generalized eigenvectors; one each of ranks 3, 2 and 1. Since $\backslash lambda\_1$ corresponds to a single chain of three linearly independent generalized eigenvectors, we know that there is a generalized eigenvector $\backslash mathbf\; x\_3$ of rank 3 corresponding to $\backslash lambda\_1$ such that

{{NumBlk|:::|$(A\; -\; 5I)^3\; \backslash mathbf\; x\_3\; =\; \backslash mathbf\; 0$|{{EquationRef|3}}}}

but

{{NumBlk|:::|$(A\; -\; 5I)^2\; \backslash mathbf\; x\_3\backslash neq\; \backslash mathbf\; 0\; .$|{{EquationRef|4}}}}

Equations ({{EquationNote|3}}) and ({{EquationNote|4}}) represent linear systems that can be solved for $\backslash mathbf\; x\_3$. Let

:$$

\mathbf x_3 =

\begin{pmatrix}

x_{31} \\

x_{32} \\

x_{33} \\

x_{34}

\end{pmatrix}.

Then

:$$

(A - 5I)^3 \mathbf x_3 =

\begin{pmatrix}

0 & 0 & 0 & 14 \\

0 & 0 & 0 & -4 \\

0 & 0 & 0 & 3 \\

0 & 0 & 0 & -1

\end{pmatrix}

\begin{pmatrix}

x_{31} \\

x_{32} \\

x_{33} \\

x_{34}

\end{pmatrix} =

\begin{pmatrix}

14 x_{34} \\

-4 x_{34} \\

3 x_{34} \\

- x_{34}

\end{pmatrix} =

\begin{pmatrix}

0 \\

0 \\

0 \\

0

\end{pmatrix}

and

:$$

(A - 5I)^2 \mathbf x_3 =

\begin{pmatrix}

0 & 0 & 2 & -8 \\

0 & 0 & 0 & 4 \\

0 & 0 & 0 & -3 \\

0 & 0 & 0 & 1

\end{pmatrix}

\begin{pmatrix}

x_{31} \\

x_{32} \\

x_{33} \\

x_{34}

\end{pmatrix} =

\begin{pmatrix}

2 x_{33} - 8 x_{34} \\

4 x_{34} \\

-3 x_{34} \\

x_{34}

\end{pmatrix} \ne

\begin{pmatrix}

0 \\

0 \\

0 \\

0

\end{pmatrix}.

Thus, in order to satisfy the conditions ({{EquationNote|3}}) and ({{EquationNote|4}}), we must have $x\_\{34\}\; =\; 0$ and $x\_\{33\}\; \backslash ne\; 0$. No restrictions are placed on $x\_\{31\}$ and $x\_\{32\}$. By choosing $x\_\{31\}\; =\; x\_\{32\}\; =\; x\_\{34\}\; =\; 0,\; x\_\{33\}\; =\; 1$, we obtain

:$$

\mathbf x_3 =

\begin{pmatrix}

0 \\

0 \\

1 \\

0

\end{pmatrix}

as a generalized eigenvector of rank 3 corresponding to $\backslash lambda\_1\; =\; 5$. Note that it is possible to obtain infinitely many other generalized eigenvectors of rank 3 by choosing different values of $x\_\{31\}$, $x\_\{32\}$ and $x\_\{33\}$, with $x\_\{33\}\; \backslash ne\; 0$. Our first choice, however, is the simplest.{{harvtxt|Bronson|1970|pp=190–191}}

Now using equations ({{EquationNote|1}}), we obtain $\backslash mathbf\; x\_2$ and $\backslash mathbf\; x\_1$ as generalized eigenvectors of rank 2 and 1, respectively, where

:$$

\mathbf x_2 = (A - 5I) \mathbf x_3 =

\begin{pmatrix}

-2 \\

2 \\

0 \\

0

\end{pmatrix},

and

:$$

\mathbf x_1 = (A - 5I) \mathbf x_2 =

\begin{pmatrix}

2 \\

0 \\

0 \\

0

\end{pmatrix}.

The simple eigenvalue $\backslash lambda\_2\; =\; 4$ can be dealt with using standard techniques and has an ordinary eigenvector

:$$

\mathbf y_1 =

\begin{pmatrix}

-14 \\

4 \\

-3 \\

1

\end{pmatrix}.

A canonical basis for $A$ is

:$$

\left\{ \mathbf x_3, \mathbf x_2, \mathbf x_1, \mathbf y_1 \right\} =

\left\{

\begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix}

\begin{pmatrix} -2 \\ 2 \\ 0 \\ 0 \end{pmatrix}

\begin{pmatrix} 2 \\ 0 \\ 0 \\ 0 \end{pmatrix}

\begin{pmatrix} -14 \\ 4 \\ -3 \\ 1 \end{pmatrix}

\right\}.

$\backslash mathbf\; x\_1,\; \backslash mathbf\; x\_2$ and $\backslash mathbf\; x\_3$ are generalized eigenvectors associated with $\backslash lambda\_1$, while $\backslash mathbf\; y\_1$ is the ordinary eigenvector associated with $\backslash lambda\_2$.

This is a fairly simple example. In general, the numbers $\backslash rho\_k$ of linearly independent generalized eigenvectors of rank $k$ will not always be equal. That is, there may be several chains of different lengths corresponding to a particular eigenvalue.{{harvtxt|Bronson|1970|pp=197–198}}

{{Main|Generalized modal matrix}}

Let $A$ be an n × n matrix. A generalized modal matrix $M$ for $A$ is an n × n matrix whose columns, considered as vectors, form a canonical basis for $A$ and appear in $M$ according to the following rules:

• All Jordan chains consisting of one vector (that is, one vector in length) appear in the first columns of $M$.
• All vectors of one chain appear together in adjacent columns of $M$.
• Each chain appears in $M$ in order of increasing rank (that is, the generalized eigenvector of rank 1 appears before the generalized eigenvector of rank 2 of the same chain, which appears before the generalized eigenvector of rank 3 of the same chain, etc.).{{harvtxt|Bronson|1970|p=205}}

{{Main|Jordan normal form}}

Let $V$ be an n-dimensional vector space; let $\backslash phi$ be a linear map in {{math|L(V)}}, the set of all linear maps from $V$ into itself; and let $A$ be the matrix representation of $\backslash phi$ with respect to some ordered basis. It can be shown that if the characteristic polynomial $f(\backslash lambda)$ of $A$ factors into linear factors, so that $f(\backslash lambda)$ has the form

:$f(\backslash lambda)\; =\; \backslash pm\; (\backslash lambda\; -\; \backslash lambda\_1)^\{\backslash mu\_1\}(\backslash lambda\; -\; \backslash lambda\_2)^\{\backslash mu\_2\}\; \backslash cdots\; (\backslash lambda\; -\; \backslash lambda\_r)^\{\backslash mu\_r\}\; ,$

where $\backslash lambda\_1,\; \backslash lambda\_2,\; \backslash ldots\; ,\; \backslash lambda\_r$ are the distinct eigenvalues of $A$, then each $\backslash mu\_i$ is the algebraic multiplicity of its corresponding eigenvalue $\backslash lambda\_i$ and $A$ is similar to a matrix $J$ in Jordan normal form, where each $\backslash lambda\_i$ appears $\backslash mu\_i$ consecutive times on the diagonal, and the entry directly above each $\backslash lambda\_i$ (that is, on the superdiagonal) is either 0 or 1: in each block the entry above the first occurrence of each $\backslash lambda\_i$ is always 0 (except in the first block); all other entries on the superdiagonal are 1. All other entries (that is, off the diagonal and superdiagonal) are 0. (But no ordering is imposed among the eigenvalues, or among the blocks for a given eigenvalue.) The matrix $J$ is as close as one can come to a diagonalization of $A$. If $A$ is diagonalizable, then all entries above the diagonal are zero.{{harvtxt|Beauregard|Fraleigh|1973|p=311}} Note that some textbooks have the ones on the subdiagonal, that is, immediately below the main diagonal instead of on the superdiagonal. The eigenvalues are still on the main diagonal.{{harvtxt|Cullen|1966|p=114}}{{harvtxt|Franklin|1968|p=122}}

Every n × n matrix $A$ is similar to a matrix $J$ in Jordan normal form, obtained through the similarity transformation $J\; =\; M^\{-1\}AM$, where $M$ is a generalized modal matrix for $A$.{{harvtxt|Bronson|1970|p=207}} (See Note above.)

Find a matrix in Jordan normal form that is similar to

:$$

A =

\begin{pmatrix}

0 & 4 & 2 \\

-3 & 8 & 3 \\

4 & -8 & -2

\end{pmatrix}.

Solution: The characteristic equation of $A$ is $(\backslash lambda\; -\; 2)^3\; =\; 0$, hence, $\backslash lambda\; =\; 2$ is an eigenvalue of algebraic multiplicity three. Following the procedures of the previous sections, we find that

:$\backslash operatorname\{rank\}(A\; -\; 2I)\; =\; 1$

and

:$\backslash operatorname\{rank\}(A\; -\; 2I)^2\; =\; 0\; =\; n\; -\; \backslash mu\; .$

Thus, $\backslash rho\_2\; =\; 1$ and $\backslash rho\_1\; =\; 2$, which implies that a canonical basis for $A$ will contain one linearly independent generalized eigenvector of rank 2 and two linearly independent generalized eigenvectors of rank 1, or equivalently, one chain of two vectors $\backslash left\backslash \{\; \backslash mathbf\; x\_2,\; \backslash mathbf\; x\_1\; \backslash right\backslash \}$ and one chain of one vector $\backslash left\backslash \{\; \backslash mathbf\; y\_1\; \backslash right\backslash \}$. Designating , we find that

:$$

M =

\begin{pmatrix}

2 & 2 & 0 \\

1 & 3 & 0 \\

0 & -4 & 1

\end{pmatrix},

and

:$$

J =

\begin{pmatrix}

2 & 0 & 0 \\

0 & 2 & 1 \\

0 & 0 & 2

\end{pmatrix},

where $M$ is a generalized modal matrix for $A$, the columns of $M$ are a canonical basis for $A$, and $AM\; =\; MJ$.{{harvtxt|Bronson|1970|pp=208}} Note that since generalized eigenvectors themselves are not unique, and since some of the columns of both $M$ and $J$ may be interchanged, it follows that both $M$ and $J$ are not unique.{{harvtxt|Bronson|1970|p=206}}

In Example 3, we found a canonical basis of linearly independent generalized eigenvectors for a matrix $A$. A generalized modal matrix for $A$ is

:$$

M =

\begin{pmatrix} \mathbf y_1 & \mathbf x_1 & \mathbf x_2 & \mathbf x_3 \end{pmatrix} =

\begin{pmatrix}

-14 & 2 & -2 & 0 \\

4 & 0 & 2 & 0 \\

-3 & 0 & 0 & 1 \\

1 & 0 & 0 & 0

\end{pmatrix}.

A matrix in Jordan normal form, similar to $A$ is

:$J\; =\; \backslash begin\{pmatrix\}$

4 & 0 & 0 & 0 \\

0 & 5 & 1 & 0 \\

0 & 0 & 5 & 1 \\

0 & 0 & 0 & 5

\end{pmatrix},

so that $AM\; =\; MJ$.

{{Main|Matrix function}}

Three of the most fundamental operations which can be performed on square matrices are matrix addition, multiplication by a scalar, and matrix multiplication.{{harvtxt|Beauregard|Fraleigh|1973|pp=57–61}} These are exactly those operations necessary for defining a polynomial function of an n × n matrix $A$.{{harvtxt|Bronson|1970|p=104}} If we recall from basic calculus that many functions can be written as a Maclaurin series, then we can define more general functions of matrices quite easily.{{harvtxt|Bronson|1970|p=105}} If $A$ is diagonalizable, that is

:$D\; =\; M^\{-1\}AM\; ,$

with

:$$

D =

\begin{pmatrix}

\lambda_1 & 0 & \cdots & 0 \\

0 & \lambda_2 & \cdots & 0 \\

\vdots & \vdots & \ddots & \vdots \\

0 & 0 & \cdots & \lambda_n

\end{pmatrix},

then

:$$

D^k =

\begin{pmatrix}

\lambda_1^k & 0 & \cdots & 0 \\

0 & \lambda_2^k & \cdots & 0 \\

\vdots & \vdots & \ddots & \vdots \\

0 & 0 & \cdots & \lambda_n^k

\end{pmatrix}

and the evaluation of the Maclaurin series for functions of $A$ is greatly simplified.{{harvtxt|Bronson|1970|p=184}} For example, to obtain any power k of $A$, we need only compute $D^k$, premultiply $D^k$ by $M$, and postmultiply the result by $M^\{-1\}$.{{harvtxt|Bronson|1970|p=185}}

Using generalized eigenvectors, we can obtain the Jordan normal form for $A$ and these results can be generalized to a straightforward method for computing functions of nondiagonalizable matrices.{{harvtxt|Bronson|1970|pp=209–218}} (See Matrix function#Jordan decomposition.)

{{Main|Ordinary differential equation}}

Consider the problem of solving the system of linear ordinary differential equations

{{NumBlk|::|$\backslash mathbf\; x\text{'}\; =\; A\; \backslash mathbf\; x\; ,$|{{EquationRef|5}}}}

where

:$$

\mathbf x =

\begin{pmatrix}

x_1(t) \\

x_2(t) \\

\vdots \\

x_n(t)

\end{pmatrix}, \quad

\mathbf x' =

\begin{pmatrix}

x_1'(t) \\

x_2'(t) \\

\vdots \\

x_n'(t)

\end{pmatrix},

{{spaces|4}} and {{spaces|4}} $A\; =\; (a\_\{ij\})\; .$

If the matrix $A$ is a diagonal matrix so that $a\_\{ij\}\; =\; 0$ for $i\; \backslash ne\; j$, then the system ({{EquationNote|5}}) reduces to a system of n equations which take the form

{{NumBlk|::|

$x\_1\text{'}\; =\; a\_\{11\}\; x\_1$

$x\_2\text{'}\; =\; a\_\{22\}\; x\_2$

:$\backslash vdots$

$x\_n\text{'}\; =\; a\_\{nn\}\; x\_n\; .$

|{{EquationRef|6}}}}

In this case, the general solution is given by

:$x\_1\; =\; k\_1\; e^\{a\_\{11\}t\}$

:$x\_2\; =\; k\_2\; e^\{a\_\{22\}t\}$

::$\backslash vdots$

:$x\_n\; =\; k\_n\; e^\{a\_\{nn\}t\}\; .$

In the general case, we try to diagonalize $A$ and reduce the system ({{EquationNote|5}}) to a system like ({{EquationNote|6}}) as follows. If $A$ is diagonalizable, we have $D\; =\; M^\{-1\}AM$, where $M$ is a modal matrix for $A$. Substituting $A\; =\; MDM^\{-1\}$, equation ({{EquationNote|5}}) takes the form $M^\{-1\}\; \backslash mathbf\; x\text{'}\; =\; D(M^\{-1\}\; \backslash mathbf\; x)$, or

{{NumBlk|::|$\backslash mathbf\; y\text{'}\; =\; D\; \backslash mathbf\; y\; ,$|{{EquationRef|7}}}}

where

{{NumBlk|::|$\backslash mathbf\; x\; =\; M\; \backslash mathbf\; y\; .$|{{EquationRef|8}}}}

The solution of ({{EquationNote|7}}) is

:$y\_1\; =\; k\_1\; e^\{\backslash lambda\_1\; t\}$

:$y\_2\; =\; k\_2\; e^\{\backslash lambda\_2\; t\}$

::$\backslash vdots$

:$y\_n\; =\; k\_n\; e^\{\backslash lambda\_n\; t\}\; .$

The solution $\backslash mathbf\; x$ of ({{EquationNote|5}}) is then obtained using the relation ({{EquationNote|8}}).{{harvtxt|Beauregard|Fraleigh|1973|pp=274–275}}

On the other hand, if $A$ is not diagonalizable, we choose $M$ to be a generalized modal matrix for $A$, such that $J\; =\; M^\{-1\}AM$ is the Jordan normal form of $A$. The system $\backslash mathbf\; y\text{'}\; =\; J\; \backslash mathbf\; y$ has the form

{{NumBlk|::|

\begin{align}

y_1' & = \lambda_1 y_1 + \epsilon_1 y_2 \\

& \vdots \\

y_{n-1}' & = \lambda_{n-1} y_{n-1} + \epsilon_{n-1} y_n \\

y_n' & = \lambda_n y_n ,

\end{align}

|{{EquationRef|9}}}}

where the $\backslash lambda\_i$ are the eigenvalues from the main diagonal of $J$ and the $\backslash epsilon\_i$ are the ones and zeros from the superdiagonal of $J$. The system ({{EquationNote|9}}) is often more easily solved than ({{EquationNote|5}}). We may solve the last equation in ({{EquationNote|9}}) for $y\_n$, obtaining $y\_n\; =\; k\_n\; e^\{\backslash lambda\_n\; t\}$. We then substitute this solution for $y\_n$ into the next to last equation in ({{EquationNote|9}}) and solve for $y\_\{n-1\}$. Continuing this procedure, we work through ({{EquationNote|9}}) from the last equation to the first, solving the entire system for $\backslash mathbf\; y$. The solution $\backslash mathbf\; x$ is then obtained using the relation ({{EquationNote|8}}).{{harvtxt|Beauregard|Fraleigh|1973|p=317}}

Lemma:

Given the following chain of generalized eigenvectors of length $r,$

:$X\_1\; =\; v\_1e^\{\backslash lambda\; t\}$

:$X\_2\; =\; (tv\_1+v\_2)e^\{\backslash lambda\; t\}$

:$X\_3\; =\; \backslash left(\backslash frac\{t^2\}\{2\}v\_1+tv\_2+v\_3\backslash right)e^\{\backslash lambda\; t\}$

:$\backslash vdots$

:$X\_r\; =\; \backslash left(\backslash frac\{t^\{r-1\}\}\{(r-1)!\}v\_1+...+\backslash frac\{t^2\}\{2\}v\_\{r-2\}+tv\_\{r-1\}+v\_r\backslash right)e^\{\backslash lambda\; t\}$,

these functions solve the system of equations,

:$X\text{'}\; =\; AX.$

Proof:

Define

:$v\_0=0$

:$X\_j(t)=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}\; v\_i.$

Then, as $\{t^\{0\}\}=1$ and $1\text{'}=0$,

:$X\text{'}\_j(t)=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 1\}^\{j-1\}\backslash frac\{t^\{j-i-1\}\}\{(j-i-1)!\}v\_i+e^\{\backslash lambda\; t\}\backslash lambda\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}v\_i$.

On the other hand we have, $v\_0=0$ and so

:$AX\_j(t)=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}Av\_i$

:$=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}(v\_\{i-1\}+\backslash lambda\; v\_i)$

:$=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 2\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}v\_\{i-1\}+e^\{\backslash lambda\; t\}\backslash lambda\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}v\_i$

:$=e^\{\backslash lambda\; t\}\backslash sum\_\{i\; =\; 1\}^\{j-1\}\backslash frac\{t^\{j-i-1\}\}\{(j-i-1)!\}v\_\{i\}+e^\{\backslash lambda\; t\}\backslash lambda\backslash sum\_\{i\; =\; 1\}^j\backslash frac\{t^\{j-i\}\}\{(j-i)!\}v\_i$

:$=X\text{'}\_j(t)$

as required.

{{reflist|30em}}

• {{ citation | first1 = Howard | last1 = Anton | year = 1987 | isbn = 0-471-84819-0 | title = Elementary Linear Algebra | edition = 5th | publisher = Wiley | location = New York }}
• {{Cite book

| last = Axler

| first = Sheldon

| title = Linear Algebra Done Right

| publisher = Springer

| year = 1997

| edition = 2nd

| isbn = 978-0-387-98258-8}}

• {{citation | first1 = Raymond A. | last1 = Beauregard | first2 = John B. | last2 = Fraleigh | year = 1973 | isbn = 0-395-14017-X | title = A First Course In Linear Algebra: with Optional Introduction to Groups, Rings, and Fields | publisher = Houghton Mifflin Co. | location = Boston | url-access = registration | url = https://archive.org/details/firstcourseinlin0000beau }}
• {{ citation | first1 = Richard | last1 = Bronson | year = 1970 | lccn = 70097490 | title = Matrix Methods: An Introduction | publisher = Academic Press | location = New York }}
• {{citation | first1 = Richard L. | last1 = Burden | first2 = J. Douglas | last2 = Faires | year = 1993 | isbn = 0-534-93219-3 | title = Numerical Analysis | edition = 5th | publisher = Prindle, Weber and Schmidt | location = Boston | url = https://archive.org/details/numericalanalysi00burd }}
• {{ citation | first1 = Charles G. | last1 = Cullen | title = Matrices and Linear Transformations | location = Reading | publisher = Addison-Wesley | year = 1966 | lccn = 66021267 }}
• {{ citation | first1 = Joel N. | last1 = Franklin | title = Matrix Theory | location = Englewood Cliffs | publisher = Prentice-Hall | year = 1968 | lccn = 68016345 }}
• {{ citation | first1 = Gene H. | last1 = Golub | first2 = Charles F. | last2 = Van Loan | year = 1996 | isbn = 0-8018-5414-8 | title = Matrix Computations | edition = 3rd | publisher = Johns Hopkins University Press | location = Baltimore }}
• {{ citation | first1 = Charlie | last1 = Harper | year = 1976 | isbn = 0-13-487538-9 | title = Introduction to Mathematical Physics | publisher = Prentice-Hall | location = New Jersey }}
• {{ citation | first1 = I. N. | last1 = Herstein | year = 1964 | isbn = 978-1114541016 | title = Topics In Algebra | publisher = Blaisdell Publishing Company | location = Waltham }}
• {{citation | first1 = Erwin | last1 = Kreyszig | year = 1972 | isbn = 0-471-50728-8 | title = Advanced Engineering Mathematics | edition = 3rd | publisher = Wiley | location = New York | url = https://archive.org/details/advancedengineer00krey }}
• {{ citation | first1 = Evar D. | last1 = Nering | year = 1970 | title = Linear Algebra and Matrix Theory | edition = 2nd | publisher = Wiley | location = New York | lccn = 76091646 }}

{{Linear algebra|collapsed}}

{{Areas of mathematics|collapsed}}

Category:Linear algebra

Category:Matrix theory