The Spectral Representation of a Symmetric Matrix 2.3 2001/10/23 2002/07/10 00:00:00.003 GMT-5 Steven Cox cox@rice.edu Steven Cox cox@rice.edu Jacob Grabczewski jgrab@owlnet.rice.edu Benjamin Fite bfite@rice.edu spectral representation pseudo-inverse Hermitian symmetric matrix orthogonal projections Gram-Schmidt diagonal matrix Hermitian transposes and the spectral representation ofsymettric matrices are explained.

Introduction Our goal is to show that if B is symmetric then each λ j is real, each P j is symmetric and each D j vanishes. Let us begin with an example. The transfer function of B 1 1 1 1 1 1 1 1 1 is R s 1 s s 3 s 2 1 1 1 s 2 1 1 1 s 2 R s 1 s 23 -13 -13 -13 23 -13 -13 -13 -13 1 s 3 13 13 13 13 13 13 13 13 13 R s 1 s λ 1 P 1 1 s λ 2 P 2 and so indeed each of the bullets holds true. With each of the D j falling by the wayside you may also expect that the respective geometric and algebraic multiplicities coincide.

The Spectral Representation We have amassed anecdotal evidence in support of the claim that each D j in the spectral representation B j 1 h λ j P j j 1 h D j is the zero matrix when B is symmetric, i.e., when B B , or, more generally, when B B where B B Matrices for which B B are called Hermitian. Of course real symmetric matrices are Hermitian. Taking the conjugate transpose throughout we find B j 1 h λ j P j j 1 h D j That is, the λ j are the eigenvalues of B with corresponding projections P j and nilpotents D j Hence, if B B , we find on equating terms that λ j λ j P j P j and D j D j The former states that the eigenvalues of an Hermitian matrix are real. Our main concern however is with the consequences of the latter. To wit, notice that for arbitrary x , D j m j 1 x 2 x D j m j 1 D j m j 1 x D j m j 1 x 2 x D j m j 1 D j m j 1 x D j m j 1 x 2 x D j m j 2 D j m j x D j m j 1 x 2 0 As D j m j 1 x 0 for every x it follows (recall this previous exercise) that D j m j 1 0 . Continuing in this fashion we find D j m j 2 0 and so, eventually, D j 0 . If, in addition, B is real then as the eigenvalues are real and all the D j vanish, the P j must also be real. We have now established If B is real and symmetric then B j 1 h λ j P j where the λ j are real and the P j are real orthogonal projections that sum to the identity and whose pairwise products vanish. One indication that things are simpler when using the spectral representation is B 100 j 1 h λ j 100 P j As this holds for all powers it even holds for power series. As a result, B j 1 h λ j P j It is also extremely useful in attempting to solve B x b for x . Replacing B by its spectral representation and b by I b or, more to the point by j j P j b we find j 1 h λ j P j x j 1 h P j b Multiplying through by P 1 gives λ 1 P 1 x P 1 b or P 1 x P 1 b λ 1 . Multiplying through by the subsequent P j 's gives P j x P j b λ j . Hence, x j 1 h P j x j 1 h 1 λ j P j b We clearly run in to trouble when one of the eigenvalues vanishes. This, of course, is to be expected. For a zero eigenvalue indicates a nontrivial null space which signifies dependencies in the columns of B and hence the lack of a unique solution to B x b . Another way in which may be viewed is to note that, when B is symmetric, this previous equation takes the form z I B j 1 h 1 z λ j P j Now if 0 is not an eigenvalue we may set z 0 in the above and arrive at B j 1 h 1 λ j P j Hence, the solution to B x b is x B b j 1 h 1 λ j P j b as in . With we have finally reached a point where we can begin to define an inverse even for matrices with dependent columns, i.e., with a zero eigenvalue. We simply exclude the offending term in . Supposing that λ h 0 we define the pseudo-inverse of B to be B + j 1 h 1 1 λ j P j Let us now see whether it is deserving of its name. More precisely, when b ℛ B we would expect that x B + b indeed satisfies B x b . Well B B + b B j 1 h 1 1 λ j P j b j 1 h 1 1 λ j B P j b j 1 h 1 1 λ j λ j P j b j 1 h 1 P j b It remains to argue that the latter sum really is b . We know that b b B b j 1 h P j b The latter informs us that ⊥ b N B . As B B , we have, in fact, that ⊥ b N B . As P h is nothing but orthogonal projection onto N B it follows that P h b 0 and so B B + b b , that is, x B + b is a solution to B x b . The representation is unarguably terse and in fact is often written out in terms of individual eigenvectors. Let us see how this is done. Note that if x P 1 then x P 1 x and so, B x B P 1 x j 1 h λ j P j P 1 x λ 1 P 1 x λ 1 x i.e., x is an eigenvector of B associated with λ 1 . Similarly, every (nonzero) vector in P j is an eigenvector of B associated with λ j . Next let us demonstrate that each element of P j is orthogonal to each element of P k when j k . If x P j and y P k then x y P j x P k y x P j P k y 0 With this we note that if x j , 1 x j , 2 … x j , n j constitutes a basis for P j then in fact the union of such bases, x j , p 1 j h 1 p n j forms a linearly independent set. Notice now that this set has j 1 h n j elements. That these dimensions indeed sum to the ambient dimension, n , follows directly from the fact that the underlying P j sum to the n -by- n identity matrix. We have just proven If B is real and symmetric and n -by- n , then B has a set of n linearly independent eigenvectors. Getting back to a more concrete version of we now assemble matrices from the individual bases E j x j , 1 x j , 2 … x j , n j and note, once again, that P j E j E j E j E j , and so B j 1 h λ j E j E j E j E j I understand that you may feel a little overwhelmed with this formula. If we work a bit harder we can remove the presence of the annoying inverse. What I mean is that it is possible to choose a basis for each P j for which each of the corresponding E j satisfy E j E j I As this construction is fairly general let us devote a separate section to it (see Gram-Schmidt Orthogonalization).