Homework Applied Linear Algebra 2 - University of Cincinnati

Homework Applied Linear Algebra 2

Strang 4e, Winter 2012

ü assignment 1

5.5 # 11, 14, 30, 33

5.5.11. Write P, Q, R in the form l1 x1 x1H + l2 x2 x2H where

1 1 01 3 4
2 1 0 , R = 4 -3 .
P= 1 2 ,Q =
1

22

5.5.14. In the list below, which of the classes of matrices contain A and which contain B?

0100 1111

A= 0010 and B = 1 1111
0001 4 1111.

1000 1111

Orthogonal, Invertible, projection, permutation, Hermitian, rank - 1, diagonalizable, Markov. Find the eigenvalues
of A and B.

5.5.30. To which classes of matrices does P belong: orthogonal, invertible, Hermitian, unitary, factorizable into L U,
factorizable into Q R?

010
P= 0 0 1 .

100

5.5.33. Write down the 3 by 3 nt matrixcircula C = 2 I + 5 P + 4 P2. It has the same eigenvectors as P in problem
30. Find its eigenvalues.

ü Assignment 2 Fast Fourier Transform

3.5 # 1, 2, 22.

3.5.1. What are F2 and F4 for the 4 × 4 Fourier matrix F?

3.5.2. Find a permutation P of the columns of F that produces F P = F. Combine with F F = n I to find F2 and F4
for the n µ n Fourier matrix F. Do this for n = 2N.

3.5.22. How could you quickly compute these four components of FC starting from
c0 + c2, c0 - c2 , c1 + c3, c1 - c3 ? You are find the fact Fourier Transform

c0 + c2, c0 - c2 , c1 + c3, c1 - c3 ? You are find the fact Fourier Transform

FC = c0 + c1 + c2 + c3
c0 + Â c1 + Â2 c2 + Â3 c3
c0 + Â2 c1 + Â4 c2 + Â6 c3
c0 + Â3 c1 + Â6 c2 + Â9 c3

ü Assignment 3 Similarity

5.5 #45, 50

5.6 # 4

2 Hwlistapplinalg12.nb

5.5.45. If uH u = 1, show that 1 - 2 u uH is Hermitian and also unitary. The rank 1 matrix is the projection onto what
line in Cn?

5.5.50. A matrix with orthonormal eigenvectors has the form A = U L U-1 = U L UH. Prove that AH A = A AH.
These are exactly the normal matrices.

2100

5.6.4. Find a diagonal matrix M, made up of 1’s and -1’s, to show that A = 1210 and
0121

0012

2 -1 0 0

B= -1 2 -1 0 are similar.
0 -1 2 -1

0 0 -1 2

ü Assignment 4 Normal Matrices

5.6 # 19, 20

5.6.19. Suppose that T is a 3 by 3 upper triangular matrix, with entries ti j. Compare the entries of T TH and TH T, and
show that if they are equal, then T must be diagonal. All normal triangular matrices are triangular.
5.6.20. If N is normal, show that °N x¥ = ±N H xµ for every vector x. Deduce that the ith row of N has the same
leghth as the i th column. Note: If N is also upper triangular, this leads again to the conclusion that it must be diagonal
ü assignment 5 Jordan Form

5.6 # 42, 43

Appendix B # 5, 6

5.6.42. Show that A B and B A have the same eigenvalues. (Take A and B to be square matrices. A more general
statement is given in the next problem 5.6.43.)

5.6.43. If A is 6 by 4 and B is 4 by 6, then A B and B A have different sizes. Nevertheless,

I -A AB 0 I A 0 0
B 0 0 I = B B A = G.
0I

a) What the sizes of the blocks of G? They are the same in each matrix.

b) This equation is M-1 FM = G, so F and G have the same 10 eigenvalues. F has the eigenvalues of A B plus 4
0’s and G has the eigenvalues of B A with 4 0’s. So A B snf B A have the same eigenvalues plus zeros.

123 11

B.5. Find “by inspection” the Jordan forms of A = 045 and B = -1 -1 .
006

B.6. Find the Jordan form J and the matrix M for A and B (B has eigenvalues 1, 1, 1, -1. What is the solution to
d u ê dt = A u, and what is ‰A t?

ü assignment 6 due Jan. 26

5.6 # 30

6.1# 2a, b, c

5.6.30. Show that A and B are similar by finding M with B = M-1 A M.

a. A= 10 and B = 01
10 01

b. A= 11 and B = 1 -1
11 -1 1

Hwlistapplinalg12.nb 3

b. A= 11 and B = 1 -1
11 -1 1

c. A= 12 and B = 43
34 21

6.1.2. Decide for or against the positive definiteness of these matrices and write out the f = xT A x.

aL 1 3 , bL 1 -1 , cL 2 3 , dL -1 2
3 5 -1 1 3 5 2 -8

ü assignment 7 due Jan. 28

6.1 # 7 a, b, c, 14 a, b, c, 16, 20

6.2 #1

6.1.7 a. What 3 by 3 symmetric matrices A1 and A2 that correspond to f1 and f2?

f1 = x21 + x22 + x32 - 2 x1 x2 - 2 x1 x3 + 2 x2 x3

f2 = x12 + 2 x22 + 11 x23 - 2 x1 x2 - 2 x1 x3 - 4 x2 x3.

b. Show that f1 is a single perfect square and not positive definite. Where is f1 equal to 0?

c. Factor A2 into LT L. Write f2 = xT A2 x as the sum of 3 squares.

6.1.14. Which of the matrices has two positive eigenvalues? Test a > 0, a c - b2 > 0. Don’t compute the eigenval-
ues. Find an x with xT A1 x < 0.

5 6 -1 -2 1 10 1 10
A1 = 6 7 , A2 = -2 -5 , A3 = 10 100 , A4 = 10 101

6.1.16. Show that f Hx, yL = x2 + 4 x y + 3 y2 does not have a minimum at (0, 0) even though it has positive coeffi-
cients. Write f as the difference of squares and find a point where f is negative.

6.1.20. For F1Hx, yL = 1 x4 + x2 y + y2 and F2Hx, yL = x3 - xy - x, find the second derivative matrices A1 and A2:
4

¶∂2 F ë ¶∂ x2 ¶∂2 F ë ¶∂ x ¶∂ y
A= .

¶∂2 F ë ¶∂ x ¶∂ y ¶∂2 F ë ¶∂ y2

A1 is positive definite so F1 is concave up (= conves). Find the minimum point for F1 and the saddle point for F2
(where the derivatives are 0).

6.2.1. For what range of numbers a and b are the matrices A and B positive definite?

a22 124

A= 2 a 2 ,B= 2 b 8

22a 487

ü assignment 8 due Jan. 30

6.2 #2

6.3 #1, 2, 6, 9

6.2.2. Decide for or against the positive definiteness of

2 -1 -1 2 -1 -1 0122

A = -1 2 -1 , B = -1 2 1 , C = 1 0 1 .

-1 -1 2 -1 1 2 210

6.3.1. Compute AT A and its eigenvalues s12, 0 and unit eigenvectors v1, v2.

A= 1 4 .
2 8

4 Hwlistapplinalg12.nb

A= 1 4 .
2 8

6.3.2a. Compute A AT and its eigenvalues s21, 0 and unit eigenvectors u1, u2.

b). Choose the signs so that A v1 = s1 u1 and verify the SVD:

14 = H u1 u2 L s1 0 Hv1, v2LT.
28 00

c). Which four vectors give orthonormal bases for CHAL, NHAL, CIATM, NIATM?

6.3.6. Suppose that u1, …, un and v1, …, vn are orthonormal bases of Rn. Construct the matrix A that transforms
each vj into uj to give A u1 = v1, … , A un = vn.

6.3.9. Explain how U S VT expresses A as a sum of r rank 1 matrices in equation (3):

A = s1 u1 v1T + … + s1 ur vrT.
ü assignment 9 Due Feb. 1

6.2 #14, 28

6.2.14. Decide whether the following matrices are positive definite, negative definite, semidefinite, or indefinite:

123 12 0 0
2 5 4 , B=
A= 2 6 -2 0 , C = - B, D = A-1.
0 -2 5 -2
349
0 0 -2 3

Is there a real solution to - x2 - 5 y2 - 9 z2 - 4 x y - 6 x z - 8 y z = 1?

6.2.28. In the Cholesky factorization A = CT C, with C = L D , the square roots of the pivots are on the diagonal
of C. Find C (lower triangular) for

900 111

A = 0 1 2 and B = 1 2 2 .

028 127

ü assignment 10 Due Feb.3

1. Find a vector of the form a (1, 1, 2) + b (2, - 1, 1) closest to the vector (1, 1, 1).

2. Find the least squares solution of the inconsistent system of equations

x1 + x2 + x3 = 3

x3 = 1

x1 + x3 = 2

2 x1 + 5 x3 = 8

-7 x1 + 8 x2 =0

x1 + 2 x2 - x3 = 1

3a. Calculate the best linear fit of the data

x -2 -1 0 1 2 3

y 12 11 8 5 2 -3

b. Graph the data against the regression line.
c. Calculate the correlation coefficient. (You can leave this out).

ü assignment 11 Due Feb. 6

6.4 #2, 7, 8, 11, 16

6.4.2. Complete the square in P = 1 xT A x - xT b = 1 Ix - A-1 bMT AIx - A-1 bM + constant. The constant equals
2 2

Pmin because the term before it is never negative. (Why?)

Hwlistapplinalg12.nb 5

6.4.2. Complete the square in P = 1 xT A x - xT b = 1 Ix - A-1 bMT AIx - A-1 bM + constant. The constant equals
2 2

Pmin because the term before it is never negative. (Why?)

6.4.7. If B is positive definite, show from the Rayleigh quotient that the smallest eigenvalue of A + B is larger than
the smallest eigenvalue of A.

6.4.8. If l1 and m1are the smallest eigenvalues of A and B, show that the smallest eigenvalue q1 of A + B is at least as
large as l1 + m1. Try the corresponding eigenvector x in the Rayleigh quotient.

6.4.11. Find the minimum values of

RHxL = x12 - x1 x2 + x22 and RHxL = x12 - x1 x2 + x22
x12 + x22 2 x21 + x22

6.4.16. From the zero submatrix, decide on the signs of the eigenvalues of the n eigenvalues of

0 0… 0 1

0 0… 0 2

A= … … … 0 … .

0 0 … 0 n-1

1 2 … n- 1 n

ü assignment 12 Midterm Assignment

INSTRUCTIONS: Do all work on the exam pages. Many of the problems can be solved on the computer. You may
use the computer to check your answers but you should do the problems using the methods and algorithms presented
by Strang.

1. Find a matrix M and find a matrix J in Jordan form such that M-1 A M = J.

6 22
A = -2 2 0

0 02

2. Write a short essay on Jordan Form for a matrix. What is it? How do you calculate it? What good is it?

123
3. Let A = 2 4 1 .

311

a. Find the largest eigenvalue of A and the corresponding Perron vector using Rayleigh’s Method.

b. Find the eigenvalue of smallest absolute value for A by using the Rayleigh-Ritz method on A- 1.
c. Find the third eigenvalue using deflation.

HINT: Note that it might be necessary to take more than 20 iterations. You can program the Nest to stop when there
is no more movement or use the built in command FixedPoint[f, x], which will stop when there is no longer
any movement in the Nest. You can also adjust the accuracy to get a better stopping point than the default which is
probably 6 decimal places, the usual Mathematica accuracy default. Also computer arithmetic might fail to get the
null space since small nonzero numbers are often read have the same status as any other nonzero number.

4. Find A10 - 2 A5 + 10 I where A = 3 -5
1 -3 . Use the Cayley-Hamilton Theorem.

5. Let A = 04
4 6.

a. Write the quadratic form associated with A.

b. Find an orthogonal matrix Q with QT A Q = D where D is diagonal.

c. If QHXL = XT 0 4 X for X in R2, sketch G8 = 8X : QHXL = 8<.
4 6

6 Hwlistapplinalg12.nb

c. If QHXL = XT 0 4 X for X in R2, sketch G8 = 8X : QHXL = 8<.
4 6

ü assignment 13 due Feb. 13

6.5 #1, 2, 5, 6

6.5.1. Use three hat functions, with h = 1 , to solve -u²″ = 2 with uH0L = uH1L = 0.Verify the aproximation matches
4

the actual solution u = x - x2 at the nodes.

6.5.2. Solve -u²″ = x with uH0L = uH1L = 0. Then solve pproximately with two hat functions and h = 1 . Where is the

3

largest error?

6.5.5. Gelerkin's Method starts with the differential equation (say - u²″ = f HxL) instead of the energy P. The trial
solution is still u = y1 V1 + y2 V2 + … + yn Vn and the y ' s are chosen to make the difference between the - u²″ and f
orthogonal to every Vi.

Gelerkin Ÿ H-y1 V1²″ - y2 Vn²″ + … - yn Vn²″L Vj „ x = Ÿ f HxL VjHxL „ x.

Integrate the left side by parts to reach A y = f proving that the Galerkin gives some A and f as the Rayleigh-Ritz for
symmetric problems.

6.5.6. A basic identity for quadratics shows that y = A-1 b is minimizing:

PHyL = 1 yT A y - yT b = 1 Iy - A-1 bMT A Iy - A-1 bM - 1 bT A-1 b.
2 2 2

The minimum over a subspace of trial functions is at y nearest to A-1 b. (That makes the first term on the right as

small as possible; it is the key to convergence of U to u. If A = I and b = H0, 0, 1L, which multiple of V = H1, 1, 1L

gives the smallest value of PHyL = 1 yT y - y1.
2

ü assignment 14 due Feb.15

7.2 # 2, 4, 5, 6, 12

7.2.2. Which “famous” inequality gives °HA + BL x¥ § °A x¥ + °B x¥, and why does it follow from equation (5) that
°A + B¥ § °A ¥ + °B ¥?

7.2.4. For positive definite A = 2 -1 , compute ±A-1µ = 1 ê l1 and °A¥ = l2 and cHAL = l2 ê l1. Find a right-
-1 1

hand side b and a perturbation d b so that the error is the worse possible, °dx¥ ê °x¥ = c °d b¥ ê °b¥.

7.2.5. Show that if l is any eigenvector for A, A x = l x, then †l§ § °A¥.

7.2.6. The matrices in equation (4) have norms between100 and 101. Why? Equation (4): A = 1 100 ,
0 1

A-1 = 1 -100 .
0 1

7.2.12. Find the norms lmax and the condition number lmax ê lmin for these positive definite matrices:
100 0 2 1 3 1
0 2, 12, 12.

ü assignment 15

7.3 #4, 6, 7, 8, 12

7.3.4. The Markov matrix A = .9 .3 has l = 1 and .6 and the power method uk = Ak u0 converges to .75 . Find
.1 .7 .25

the eigenvectors of A-1. To what does the inverse power method u-k = IA-1Mk u0 converge (after multiplying by .6k)?

7.3.6. Compute s = °x¥, v = x + s z, and H = 1 - 2 v vT . Verify that H x = - s z: x = 3 0
4 ,z = 1.
vT v

Hwlistapplinalg12.nb 7

7.3.6. Compute s = °x¥, v=x + s z, and H = 1- 2 v vT . Verify that H x = - s z: x = 3 ,z = 0
vT v 4 1.

134
7.3.7. Using problem 6, find the tridiagonal H A H-1 similar to A = 3 1 0 .

400

7.3.8. Show that starting from A0 = 2 -1
-1 2 , the unshifted QR algorithm produces only a minor improvement

A1 = 1 14 -3
5 -3 6 .

7.3.12. Choose cos q and sin q in the rotation P to triangulize A, and find R:

cos q - sin q -1 1 * *
P21 A = sin q cos q = 0 * = R.
3 5

ü assignment 16

120 80 40 -16
80 120 16 -40
Find Householder matrices H1 and H2 so that H2 H1 A H1 H2 is tridiagonal where A = 40 16 120 -80 .
-16 -40 -80 120

ü assignment 17

120 80 40 -16
80 120 16 -40
Find Givens matrices G1,…, G6 so that G6 … G1 A such that is upper triangular where A = 40 16 120 -80 .
-16 -40 -80 120

Find Givens matrices which when multiplied on the left and right take A to tridiagonal form.

ü assignment 18

7.4 #1, 2, 4

7.4.1. The matrix has eigenvalues 2 - 2 , 2, 2 + 2 :

2 -1 0
A = -1 2 -1

0 -1 2

Find the Jacobi Matrix J = D- 1H-L - UL and the Gauss-Seidel matrix ID + LL-1 H- UL and their eigenvalues and the
numbers wopt and lmax for successive over relaxation SOR.

7.4.2. For the n by n matrix describe the Jacobi matrix J = D-1H- L - UL:

2 -1 … … …
-1 2 -1 … …
A= … … … … …
… … -1 2 -1
… … … -1 2

Show that the vector x1 = Hsin p h, sin 2 p h, …, sin n p hL is an eigenvector pf J with eigenvalue
p
l1 = cos p h = cos n + 1 .

7.4.4. The matrix

311
A= 0 4 1

225

8 Hwlistapplinalg12.nb

311
A= 0 4 1

225

is called diagonally dominated because †ai i§ > ri = ⁄j ¹≠ i °ai j• for every i. Show that 0 cannnot lie in any of the Gersh-
gorin circles and conclude that A is nonsingular.

GERSHGORIN’S THEOREM. Every eigenvalue of A lies in ‹Ci where Ci is the circle of center ai i and radius
ri = ⁄j, j¹≠ i °ai j•.

PROOF. Let x = Hxi, …, xnL be an eigenvector of A for the eigenvalue l. Then †xi§ = max °xj• is nonzero. We have that

l=

l xi = HAxLi = ⁄j ai j xj = ai1 x1 +…+ ain xn and †ai i - l§ § ¢ai1 x1 +…+ ai-1 n xi-1 + ai+1 n xi + 1 +…+ ain xn ¶ § ri
xi xi xi xi xi xi xi
xi xi

ü assignment 19 due Feb.27, 2012

8.1 #1, 2, 3, 4

8.1.1. Sketch the feasible set

x + 2y ¥ 6
2x + y ¥ 6

x ¥0
y¥0

What points lie at the three corners of this set?

8.1.2. On the preceding feasible set, what is the minimum value of the cost function x + y? Draw the line
x + y = constant that first touches the feasible set. What points minimize the cost function 3 x + y and x - y?

8.1.3. Show that the feasible set constrained by

2x + 5y § 3
-3x + 8y § -5

x ¥0
y¥ 0

is empty.

8.1.4. Show that the following problem is feasible but unbounded, so it has no optimal solution:

Maximize x + y

Subject to

-3x + 2y § -1
x- y § 2
x ¥0
y¥ 0

ü assignment 20 due Feb. 29, 2012

8.2 # 1, 2, 3, 8

8.2.1 Minimize x1 + x2 - x3 subject to

2x1 - 4x2 + x3 + x4 = 4
2
3x1 + 5x2 + x3 + x5 =

x1, …, x5 ¥ 0.

Hwlistapplinalg12.nb 9

Which of the x1, x2, x3 should enter the basis, and which of the x4, x5 should leave? Compute the new pair of basic
variables and find the cost of the new corner.

N.B. The text book does not have x1, …, x5 ¥ 0. If we did not add the positivity constraints, we would expand the
problem by substituting xi = x+i - x-i , with x+i , xi- ¥ 0.

8.2.2. After the preceding simplex step, prepare for and decide on the next step.

8.2.3. In Example 4, suppose the cost is 3 x + y. With rearrangement, the cost vector is c = H0, 1, 3, 0L. Show that
r ¥ 0 and, therefore, that corner P is optimal.

8.2.8. Minimize 2 x1 + x2 subject to

x1 + x2 ¥ 4
x1 + 3x2 ¥ 12
x1 - x2 ¥ 0
x1 ¥ 0

x2 ¥ 0

ü assignment 21 due March 2, 2012

8.1 # 8

8.2 # 10

8.3 #1, 2, 3

8.1.8. In the feasible set for the General Motors problem, the nonnegativity x, y, z ¥ 0 leaves an eigth of three
dimensional space (the positive octant). How is this cut by the two planes from the constraints, and what shape s the
feasible set? How do its corners show that, with only these two constraints, there will be only two kinds of cars in the
optimal solution.

General Motors Problem. Suppose General Motors makes a profit of $200 on each Chevrolet, $300 on each Buick,
and $500 on each Cadillac. These get 20, 17, and 14 miles per gallon, respectively, and Congress insists that the
average car must get 18. The plant can assemble a Chevrolet in 1 minute a Buick in 2 minutes and a Cadillac in 3
minutes. What is the maximum profit in 8 hours (480 minutes)?

Problem. Maximize 200 x + 300 y + 500 z

Subject to

20 x + 17 y + 14 z ¥ 18 Hx + y + zL

x + 2 y + 3 z § 480

x, y, z ¥ 0

8.2.1. Suppose we want to minmize c x = x1 - x2, subject to

2 x1 - 4x2 + x3 =6

3 x1 + 6 x2 + x4 = 12

x1, x2, x3 ¥ 0.

Starting frm H0, 0, 6, 12L should x1 or x2 be increased from its current value of zero? How far can t be increased until
the equations force x3 or x4 down to 0? At that point, what is the new x?

8.3.1. What is the dual of the following problem:

Minimize x1 + x2
subject to

2x1 ¥ 4
x1 + 3x2 ¥ 11
x1 ¥ 0
x2 ¥ 0

10 Hwlistapplinalg12.nb

Find the solution to both this problem and its dual, and verify that minimum equals maximum.

8.3.2. What is the dual of the following problem:

Maximize y2

subject to

y1 + y2 § 3
y1 ¥ 0

y2 ¥ 0

Solve both this problem and its dual.

8.3.3. Suppose A is the identity matrix (so that m = n), and the vectors b and c are nonnegative. Explan why x* = b is
optimal in the minimal problem, find y* in the maximal problem and verify that the two values are the same. If the
first component of b is negative, what are x* and y*?

ü assignment 22 Due March 6, 2012

Solve the following problems by the dual simplex algorithm:
1a. Minimize 3 x1 + 5 x2

Subject to the constraints

-4 x1 - 5 x2 + x3 = -3

6 x1 - 6 x2 + x4 =7

x1 + 8 x2 + x5 = 20

x1, … , x5 ¥ 0.

b. Minimize 3 x1 + x2

Subject to the constraints

x1 + 3 x2 ¥ 3
x1 + 5 x2 ¥ 1
2 x1 + x2 ¥ 4
4 x1 + x2 ¥ 5

x1, x2 ¥ 0.

c. Solve the following LP by the dual simplex algorithm:

Minimize 2 x1 + x2
Subject to the constraints

2 x1 + 3 x2 ¥ 3
x1 + 5 x2 ¥ 1

2 x1 + x2 ¥ 4
4 x1 + x2 ¥ 5

x1, x2 ¥ 0.

ü assignment 23 due March 7, 2012

The problem

Minimize x21 + x22 + x23 + x42 - 2 x1 - 3 x4
Subject to the constraints

2 x1 + x2 + x3 + 4 x4 = 7

x1 + x2 + 2 x3 + x4 = 6

xi ¥ 0 for all xi

has a feasible point H2, 2, 1, 0). Find the direction of the projected negative gradient d and find a t > 0 with
f Hx + t dL < f Hx L. Check the constraints for x + t d. HINT: Use the matrix A which incorporates the positivity
constraints on the variables.

Hwlistapplinalg12.nb 11

has a feasible point H2, 2, 1, 0). Find the direction of the projected negative gradient d and find a t > 0 with
f Hx + t dL < f Hx L. Check the constraints for x + t d. HINT: Use the matrix A which incorporates the positivity
constraints on the variables.

2114

1121

A= -1 0 0 0
0 -1 0 0

0 0 -1 0

0 0 0 -1

since we need to write the positivity constraints as -xi § 0. The tight constraints are given by q = 81, 2, 6<. So the
working surface has the matrix

211 4
Aq = B = 1 1 2 1 .

0 0 0 -1

ü assignment 23 (tentative)

Show that x = H1, 1, 1, 0L and y = H1, 1, 0, 1L are feasible in the primal and dual with

0010 11

0100 11
A= 1 1 1 1 ,b = 1 ,c= 1 .

1001 13

Then after computing c x and b y, explain how you know they are optimal.