Matrix Algebra

Different types of matrices can be recognized:

• rectangular -- the number of rows does not equal the number of columns

• square -- the number of rows equals the number of rows. The trace of a square matrix is the sum of the elements X(i,i) for i = 1 to N (the size of the matrix. The values on the trace constitute the diagonal of the square matrix.

• square and symmetric -- X(i,j) = X(j,i). The matrix of correlation coefficients and the matrix of variances-covariances are square and symmetric. The trace of the correlation coefficient matrix is N. The tracee of the variance-covariance matrix is the sum of the variances.

• null matrix -- a square and symmetric matrix with all elements equal to 0.0

• identify matrix -- a square and symmetrix matrix in which X(i,i) = 1 for all i = 1 to N with all X(i,j) where i is not equal to j equal to 0.0.

• vector -- a row or a column of a matrix. A column vector has dimensions (N,1) where N is the number of rows in the matrix. A row vector has dimensions (1,M)

Matrices can be manipulated if the rules noted below are satisfied.

• Equality -- two matrices are equal if all corresponding elements are equal; the matrices must be of the same order.

• Transpose -- X' is the transpose of matrix X. The rows of X become the columns of X'. X(i,j) = X'(j,i) for all i,j.

  If X         7     6      X'    7     5
                   5     4              6    4         
  

• Addition -- matrices can be added if each matrix has the same dimensions. For A = B + C, A(i,j) = B(i,j)+C(i,j) for all i,j.

  7	6				1	5		6	1
  
  			=				+
  0	4				-1	2		1	2
  

  Thus, A(2,1) = B(2,1)+C(2,1) or 0 = -1 + 1

• Subtraction -- matrices can be subtracted if each matrix has the same dimensions. For A = B - C, A(i,j) = B(i,j)-C(i,j) for all i,j.

  8	3				9	5		1	1
  
  			=				-
  5	7				9	9		4	2
  

  Thus, A(2,1) = B(2,1)-C(2,1) or 5 = 9 - 4

• Multiplication -- two matrices (B and C) can be multiplied to produce matrix A under certain conditions. For A = BC matrix B is the prefactor and matrix C is the post factor. For B(i,j) and C(k,l) A is defined if, and only if, i = l. That is, the number of rows of the pre matrix equals the number of rows of the post; the dimensions of the product matrix A and j rows and k columns. If A=BC is defined A=CB may or may not be defined. Consider the following matrices. For which is multiplication defined and, if defined, what are the dimensions of the product matrix?

  1. B(4,4) C(5,5)

  2. B(10,1)C(1,10)

  3. B(1,10)C(10,1)

  4. B(4,3)C(3,1)

  5. B(7,5)C(5,8)

  6. B(5,8)C(7,5)

  a	b				9	5		1	1
  
  			=				*
  c	d				9	9		4	2
  

  Element a (first row and first column) of the product matrix is equal to the product of the first row of the pre times the first column of the post. Element d (second row and second column) of the product matrix is equal to the product of the second row of the pre times the second column of the post.

  a = 9*1 + 5*4 = 20
  b = 9*1 + 5*2 = 19
  c = 9*1 + 9*4 = 45
  d = 9*1 + 9*2 = 27
  

• Matrix Division -- one matrix can not be divided by another in the ordinary sense of division. A simplified introduction follows (from Davis, 1973.)

  Consider the pair of simultaneous equations:

  4X1 + 10X2 = 38
  10X1 + 30X2 = 110
  
  Let matrix A be the matrix of coefficients of the equations:
  
    4  10
  10  30
  
  Let B be the solution vector:
  
    38
  110
  
  Let X be the vector of unknowns:
  
  X1
  X2
  
  The equations can be written:
  
  AX = B
  
  If ordinary division held, then X = B/A which is undefined for matricies.
  
  The inverse of a matrix A (written as A-1)is a matrix which satisfies the relationship:
  
  A-1A = I
  
  or the product of the inverse of a matrix times the matrix equals the identify matrix I. 
   
  For the matrix A defined above show that AI (or IA) = A.  
  
  Thus, the identify matrix operates like multiplying a number by 1.0.  
  (in the above, if A is a 3 by 3 matrix then the identity matrix is a 3 by 3 
  matrix with 1s on the main diagonal and 0s in all of the other positions.
  
  If both sides of the equation are multiplied by A-1 then:
  
  AA-1X = BA-1
  
  since AA-1 = I then 
  
  IX=BA-1 or 
  
  X = BA-1.
  
  
  The vector of unknowns equals the product matrix formed by pre multiplying
   the inverse of the coefficients matrix by the constant vector.  
  Show for yourself that the dimensions of the matrices and vectors satisfy 
  the rules for multiplication.
  
  Many books and articles have been written on the general subject of finding 
  the inverse of a matrix.  In our work we will be dealing with finding 
  the inverse of a square and symmetric matrix.   
  Singular matrices have no inverse.
  

• Determinants -- a single number extracted from a square matrix. If the determinant of a square matrix is 0.0 then the matrix is singular and its inverse does not exist.

  Consider the following matrix:

    4  10
  10  30
  

  The determinant (usually written as |a| for matrix A) is given by subtracting the product of the two off the main diagonal elements (10 times 10) from the product of the elements on the main diagonal (4 times 30) or |a| = 120 - 100 or 20.

  Determinants can be computed for larger square matricies (Cramer's Rule).