UC SCI 111, Oct 30, 2002
Notebook 8
Mathematica: Matrices and vectors

Mathematica prerequisites

In Mathematica, a matrix M is represented as a list of lists. That is, every row of M has its own list, and therefore, the list of rows corresponds with M itself. The most straightforward way of defining a matrix in Mathematica is by just typing in all elements, row by row.

m= {{a,b},{c,d}}

The command MatrixForm tells Mathematica that we want m to be displayed as a matrix, rather than as a list of lists.

m// MatrixForm

Since we will be viewing several matrices in this session, let's have Mathematica put all matrix output in MatrixForm. Enter the cell below to accomplish this.

$Post:=If[MatrixQ[#],MatrixForm[#],#]&

Now when we ask for m, it will be displayed nicely.

m

We can also define a matrix using a template. Go to the Input menu, and choose "Create Table/Matrix/Palette...". In the dialog box that appears, ask for a matrix and enter the dimensions. A formatted matrix will appear on screen, like this:
			
			(({{□, □}, {□, □}}))

Use the Tab key to move from position to position to enter values into the matrix.

Similarly, vectors may be entered directly as lists, or in formatted form as n×1 column matrices. To enter a vector as a list, simply use curly brackets:

v={x,y}

To enter a vector as a column matrix, use this typing trick: Type a left parentheses "(" followed by the first coordinate of the vector. Now hit Ctrl|⌤ (where ⌤ is the carriage return) to get a new row, and enter your second coordinate. Use the right arrow to move the cursor to the right so you can finish by adding the right parentheses. See if you can create a new cell that looks just like this one:

w=(({{3}, {4}}))

Some elementary operations

Multiplication of numbers in Mathematica is indicated by  a * or by just a space, like in 2 3. Matrix multiplication however, uses a dot! Note that Mathematica treats v as a column vector, but does not display it as such.

m.v

m.w

Square matrices may be multiplied by themselves, i.e. raised to a power. However, ordinary syntax does not yield matrix multiplication:

(m^2)

Instead, find the square of a matrix by using the command MatrixPower[m,2].

MatrixPower[m,2]

Higher powers are found by increasing the value of the second argument.

In order to experiment with matrix multiplication we introduce a function which produces random matrices of any given sizes m by n.

mat[m_,n_]:=RandomInteger[{-5,5},{m,n}]

So for example

A=mat[4,3]

Exercise 1.
Explain why the following command generates an error message

A.A

An important matrix is the n by n identity matrix. Here are two examples with n=3,4

id3=IdentityMatrix[3]
id4=IdentityMatrix[4]

Verify that the following multiplications do not affect A

A.id3
id4.A

Very often an n by n matrix has an inverse which can be determined using MatrixPower

A=mat[3,3]
B=MatrixPower[A,-1]

Exercise 2.
Check that both A B and B A are equal to the 3 by 3 identity matrix

Exercise 3.
Of course a 3 by 3 matrix with only 0 as elements cannot have an inverse. Explain why. Can you think of a 3 by 3 matrix, not all elements 0, which does not have an inverse?

Matrix recurrences and eigenvectors

Consider the matrix  M=(({{-2, 2, -3}, {2, 1, -6}, {-1, -2, 0}})).

Consider the recurrence relation in (R^3) given by

x(n)=M. x(n-1)		x(0)∈(R^3).

The general solution of this recurrence is given by x(n)=(M^n). x(0) for all n. To compute (M^n) using Mathematica remember that we must use the command  MatrixPower[M,n].

Exercise 4.
Take three arbitrary different starting vectors x(0) and compute x(20) for each starting value. In each case divide x(20) by the third component of x(20) to see what the ratios of the components of x(20) are. Compute the ratios numerically! 
What do you observe?
Instead of x(20) repeat the experiment with x(40).

If all went well, you have seen in the previous exercise that the vector  v=(({{-1}, {-2}, {1}}))  plays an important role.

Exercise 5.
a) Compute M v and note that we get a multiple of v. Which multiple is that?
b) There are more vectors with similar behavior. For example v = (({{3}, {0}, {1}})). Check which multiple we get for this vector.
c) Same question as (b) but now with  v = (({{-2}, {1}, {0}})).
d) Convince yourself that the three vectors v above are special in the sense that M v is again a multiple of v. Do this by computing M v for several other choices of v.

The three vectors above are called eigenvectors of the matrix M. Mathematica has a special command to compute them.

Eigenvectors[M]

The multiplication factors are known as eigenvalues of M. They can be computed by the following command (this will be displayed as a matrix, but think of it as a list of three vectors -- the rows of the matrix.)

Eigenvalues[M]

More about recurrences, eigenvalues, and eigenvectors

Matrix multiplication has two important and easily verified properties:

1. For all vectors v and all scalars α, we have M (α v)=α M v.
2. For all vectors v and w we have M(v+w)=M v + M w.

In the context of the current discussion, these facts are useful. For suppose now that a given matrix M has an eigenvector v with eigenvalue λ (so that M v = λ v). If we were to apply M to a scalar multiple w=α v of v, we would (by 1) have
	 M w = M(α v)=α M v=α λ v=λ α  v=λ w.
And we see that w is also an eigenvector of M with eigenvalue λ. We conclude that for each eigenvalue λ, the associated eigenvector(s) are determined only up to scalar multiples. Moreover, it follows immediately that the iterates of w are (M^n)w=(λ^n)w.

Exercise 6.
Verify that for the matrix M of the last problem, if we choose  x(0)=w=(({{-6}, {0}, {-2}}))=-2(({{3}, {0}, {1}})), then x(20)=(λ^20)w, where λ is the eigenvalue associated with the eigenvector v=(({{3}, {0}, {1}})).

Even more important, however, is the following: Suppose the 3 by 3 matrix M has 3 distinct eigenvalues (Subscript[λ, 1]), (Subscript[λ, 2]), and (Subscript[λ, 3]), with corresponding eigenvectors (Subscript[v, 1]), (Subscript[v, 2]), and (Subscript[v, 3]), as does the matrix M of the previous section. And suppose you wish to track the iterates of the initial vector x(0), which is expressed a linear combination of the three eigenvectors. That is, we suppose that 
	x(0)=(Subscript[α, 1])(Subscript[v, 1])+(Subscript[α, 2])(Subscript[v, 2])+(Subscript[α, 3])(Subscript[v, 3])
for some scalars (Subscript[α, 1]), (Subscript[α, 2]), and (Subscript[α, 3]). If x(0) can be so expressed, there is an elegant and revealing expression for the iterates of x(0) under multiplication with M:

({{x(n)=(M^n)x(0)}, {=(M^n)((Subscript[α, 1])(Subscript[v, 1])+(Subscript[α, 2])(Subscript[v, 2])+(Subscript[α, 3])(Subscript[v, 3]))}, {=(Subscript[α, 1])(M^n)(Subscript[v, 1])+(Subscript[α, 2])(M^n)(Subscript[v, 2])+(Subscript[α, 3])(M^n)(Subscript[v, 3])}, {=(Subscript[α, 1])((Subscript[λ, 1])^n)(Subscript[v, 1])+(Subscript[α, 2])((Subscript[λ, 2])^n)(Subscript[v, 2])+(Subscript[α, 3])((Subscript[λ, 3])^n)(Subscript[v, 3])}, {=((Subscript[λ, 1])^n)(Subscript[α, 1])(Subscript[v, 1])+((Subscript[λ, 2])^n)(Subscript[α, 2])(Subscript[v, 2])+((Subscript[λ, 3])^n)(Subscript[α, 3])(Subscript[v, 3])}})

As an example, consider x(0)=(({{-3}, {-1}, {1}})). It is easy to verify (although, perhaps, less easy to derive) that x(0) can be expressed as a linear combination of eigenvectors of M as follows: x(0)=(({{-3}, {-1}, {1}}))=(({{-1}, {-2}, {1}}))+(({{-2}, {1}, {0}})). Recalling that the eigenvalues corresponding to these two eigenvectors are 5 and -3, respectively, we conclude that  x(n)=(5^n)(({{-1}, {-2}, {1}}))+(((-3))^n)(({{-2}, {1}, {0}})).

Exercise 7.
a) Verify the conclusion above for n=20 by calculating x(20).
b) Explain in words why for large n, the iterates of x(0) approach scalar multiples of the eigenvector (({{-1}, {-2}, {1}})).

We now have a reasonable explanation for the behavior observed in exercise 4. The only remaining question is this: Can every possible initial vector x(0) be expressed as a linear combination of the eigenvectors of the matrix M? That is, can we write any vector x(0) in the form

x(0)=(Subscript[α, 1])(({{-1}, {-2}, {1}}))+(Subscript[α, 2])(({{-2}, {1}, {0}}))+(Subscript[α, 3])(({{3}, {0}, {1}}))

for suitably chosen scalars (Subscript[α, 1]), (Subscript[α, 2]), and (Subscript[α, 3])? If so, we can reasonably conclude that so long as (Subscript[α, 1])is nonzero, the iterates of x(0) will eventually approach multiples of the vector (({{-1}, {-2}, {1}})).

We will find that for this particular matrix M, such a decomposition is always possible, and indeed we can determine the coefficients (Subscript[α, 1]), (Subscript[α, 2]), and (Subscript[α, 3]) very easily. It is here that the notion of the inverse of a matrix is needed. Why? We simply observe the decomposition above is really just matrix multiplication:

(Subscript[α, 1])(({{-1}, {-2}, {1}}))+(Subscript[α, 2])(({{-2}, {1}, {0}}))+(Subscript[α, 3])(({{3}, {0}, {1}}))=(({{-1(Subscript[α, 1])+-2(Subscript[α, 2])+3(Subscript[α, 3])}, {-2(Subscript[α, 1])+1(Subscript[α, 2])+0(Subscript[α, 3])}, {1(Subscript[α, 1])+0(Subscript[α, 2])+1(Subscript[α, 3])}}))=(({{-1, -2, 3}, {-2, 1, 0}, {1, 0, 1}}))(({{(Subscript[α, 1])}, {(Subscript[α, 2])}, {(Subscript[α, 3])}}))

If we simply multiply each side of the equation on the left by the inverse of the square matrix (assuming it exists), we will have the requisite values of (Subscript[α, 1]), (Subscript[α, 2]), and (Subscript[α, 3]). Mathematica is well suited to carrying out such a calculation.

Exercise 9.
For each of the initial values x(0) given below, express x(0) as a linear combination of the eigenvectors of M shown above (i.e. find the coefficients (Subscript[α, i])).
Based on the decomposition, make a conjecture as to the limiting direction of the higher iterates of x(0).
a) x(0)=(({{8}, {4}, {0}})).
b) x(0)=(({{1}, {1}, {1}})).

Visualisation of eigenvectors of 2 by 2 matrices

Suppose we are given a 2 by 2 matrix. For example

M=(({{0, 0.5}, {0.5, 0}}))

To visualise the action of M on the vectors in the plane we take an arbitrary vector v whose endpoint lies on the unit circle, and plot Mv by the arrow whose starting point lies in the endpoint of v. We do this for a number of choices of v which are distributed regularly over the unit circle. The program that does the plotting is given here:

<<"VectorFieldPlots`";
plotmap[M_,N_:40]:=Module[{vectors,u,g1,g2},vectors=Table[{{Cos[u],Sin[u]},M.{Cos[u],Sin[u]}},{u,0,2 π,((2 π)/N)}];g1=(Needs["VectorFieldPlots`"];ListVectorPlot[vectors,DisplayFunction->Identity]);g2=Graphics[{Hue[0.8`],Circle[{0,0},1]}];Show[g1,g2,Axes->True,Ticks->None,DisplayFunction->$DisplayFunction]]

The function plotmap takes the matrix M as its argument and a (optional) number N, which is the number of desired vectors to be plotted. The default value for N is 40. Here is our example:

plotmap[M]

The eigenvectors of M are those vectors, whose image is pointing precisely away from, or towards the origin. Please explain this, and indicate in the above plot  the eigenvectors of our chosen matrix (({{0, 1}, {1, 0}})).

Draw the plots for the matrices (({{-3, -1}, {1, 0}})), (({{1, 2}, {3, 6}})),(({{3, 1}, {1, 1}})). Do the results correspond to the results gotten by hand computation of the eigenvectors?

Repeat the above exercise, replacing the given matrices wtih their fourth (or higher) powers. Explain the resulting images (recall the work done in previous sections of this notebook).

A very simple matrix is of course the identity matrix

What are the eigenvectors of the identity matrix? Check this by making a plot of the corresponding linear map.

A very interesting situation occurs if our matrix M has eigenvalues which are complex numbers. For example the rotation matrix (({{0, -1}, {1, 0}})) corresponding to anti-clock wise rotation around 90 degrees.

a) Make a plot of this mapping and convince yourself that there are no (real) eigenvectors.
b) Suppose we add the identity matrix to the rotation matrix above. So we have the matrix M=(({{1, -1}, {1, 1}})). Can you predict what the plot of the resulting linear map will look like?  Verify your prediction.
c) Now subtract the identity matrix from the rotation matrix. So we get the matrix M=(({{-1, -1}, {1, -1}})). Answer the same questions as in part (b).
d) Compute the complex eigenvalues of the matrices in part a)b)c).

The plots until now looked rather circular, but this need not be the case.

Make a plot of the map corresponding to M=(({{1, 1}, {-1, 0}})). Compute the complex eigenvalues.

Finally there is a border case, which does not occur frequently, but if it does, it can be troublesome.

a) Make a plot of the map corresponding to M=(({{1, 1}, {0, 1}})). Compute the eigenvalues and eigenvectors.
b) Make a plot of the map corresponding to M=(({{3, 2}, {-2, -1}})). Compute the eigenvalues and eigenvectors.