Kronecker Products and Matrix Calculus with Applications

by Alexander Graham

Overview

Enhanced by many worked examples — as well as problems and solutions — this in-depth text discusses the Kronecker matrix product. Named after a 19th-century German mathematician, Leopold Kronecker, the Kronecker product is an increasingly important and useful matrix operation and an area of matrix calculus with numerous applications. A great deal of information previously only available in specialized and hard-to-find texts and papers is presented here at an undergraduate level.
After the preliminaries of an introductory chapter, the treatment proceeds to examinations of the Kronecker product and its applications. An introduction to matrix calculus is followed by further development of the subject, including an application of Kronecker products. Final chapters explore the derivative of a matrix and some applications of matrix calculus. The text concludes with solutions to the problems that appear at the end of each chapter and helpful tables of formulae and derivatives.

Product Details

ISBN-13:	9780486831206
Publisher:	Dover Publications
Publication date:	06/13/2018
Series:	Dover Books on Mathematics
Sold by:	Barnes & Noble
Format:	eBook
Pages:	144
File size:	10 MB

About the Author

Dr. Alexander Graham, now retired, was Senior Lecturer on the Faculty of Mathematics at The Open University, Milton Keynes, UK. Dover also publishes his Matrix Theory and Applications for Scientists and Engineers.

Read an Excerpt

CHAPTER 1

Preliminaries

1.1 INTRODUCTION

In this chapter we introduce some notation and discuss some results which will be found very useful for the development of the theory of both Kronecker products and matrix differentiation. Our aim will be to make the notation as simple as possible although inevitably it will be complicated. Some simplification may be obtained at the expense of generality. For example, we may show that a result holds for a square matrix of order n X n and state that it holds in the more general case when A is of order m X n. We will leave it to the interested reader to modify the proof for the more general case.

Further, we will often write

[MATHEMATICAL EXPRESSION OMITTED]

when the summation limits are obvious from the context.

Many other simplifications will be used as the opportunities arise. Unless of particular importance, we shall not state the order of the matrices considered. It will be assumed that, for example, when taking the product AB or ABC the matrices are conformable.

1.2 UNIT VECTORS AND ELEMENTARY MATRICES

[MATHEMATICAL EXPRESSION OMITTED] (1.1)

The unit vectors of order n are defined as

The one vector of order n is defined as

[MATHEMATICAL EXPRESSION OMITTED] (1.2)

From (1.1) and (1.2), obtain the relation

e = [summation]Zei (1.3)

The elementary matrixEij is defined as the matrix (of order m X n) which has a unity in the (i,j)th position and all other elements are zero.

For example,

[MATHEMATICAL EXPRESSION OMITTED] (1.4)

The relation between ei, ej and Eij is as follows

Eij = eie'j (1.5)

where e'j denotes the transposed vector (that is, the row vector) of ej.

Example 1.1

Using the unit vectors of order 3

(i) form E11, E21, and E23

(ii) write the unit matrix of order 3 X 3 as a sum of the elementary matrices.

Solution

(i) [MATHEMATICAL EXPRESSION OMITTED]

(ii) [MATHEMATICAL EXPRESSION OMITTED]

The Kronecker delta δij is defined as

[MATHEMATICAL EXPRESSION OMITTED]

it can be expressed as

δij = e'iej = e'jei (1.6)

We can now determine some relations between unit vectors and elementary matrices.

[MATHEMATICAL EXPRESSION OMITTED] (1.7)

and

[MATHEMATICAL EXPRESSION OMITTED] (1.8)

Also

[MATHEMATICAL EXPRESSION OMITTED] (1.9)

In particular if r = j, we have

EijEjs = δjjEis = Eis

and more generally

EijEjsEsm = EisEsm = Eim. (1.10)

Notice from (1.9) that

EijErs = 0 if j ≠ r.

1.3 DECOMPOSITIONS OF A MATRIX

We consider a matrix A of order m X n having the following form

[MATHEMATICAL EXPRESSION OMITTED] (1.11)

We denote the n columns of A by A.1, A.2, ... A.n. So that

[MATHEMATICAL EXPRESSION OMITTED] (1.12)

and the m rows of A by A1., A.2, ... Am. so that

[MATHEMATICAL EXPRESSION OMITTED] (1.13)

Both the A.j and the Ai. are column vectors. In this notation we can write A as the (partitioned) matrix

A = [A.1 A.2 ... A.n] (1.14)

or as

A = [A.1 A.2 ... Am.]' (1.15)

(where the prime means 'the transpose of').

For example, let

[MATHEMATICAL EXPRESSION OMITTED]

so that

[MATHEMATICAL EXPRESSION OMITTED]

then

[MATHEMATICAL EXPRESSION OMITTED]

The elements, the columns and the rows of A can be expressed in terms of the unit vectors as follows:

The jth column A.j = Aej (1.16)

The ith row Ai,' = e'iA. (1.17)

So that

Ai. = (e'iA)' = A'ei. (1.18)

The (i,j)th element of A can now be written as

aij = e'iAej = e'jA'ei. (1.19)

We can express A as the sum

A = [summation][summation]aijEij (1.20)

(where the Eij are of course of the same order as A) so that

[MATHEMATICAL EXPRESSION OMITTED] (1.21)

From (1.16) and (1.21)

[MATHEMATICAL EXPRESSION OMITTED] (1.22)

Similarly

Ai. = [summation over j]aijej (1.23)

so that 7

A'i. = [summation over j]aije'j (1.23)

It follows from (1.21), (1.22), and (1.24) that

A = [summation]A.je'j (1.25)

and

A = [summation]ejAi.'. (1.26)

Example 1.2

Write the matrix

[MATHEMATICAL EXPRESSION OMITTED]

as a sum of: (i) column vectors of A; (ii) row vectors of A.

Solutions

(i) Using (1.25)

[MATHEMATICAL EXPRESSION OMITTED]

Using (1.26)

[MATHEMATICAL EXPRESSION OMITTED]

There exist interesting relations involving the elementary matrices operating on the matrix A.

For example

[MATHEMATICAL EXPRESSION OMITTED] (1.27)

Similarly

AEij = Aeie'j = A.ie'j (by 1.16) (1.28)

so that

AEjj A.je'j (1.29)

AEijB = Aeie'jB = A.iBj'. (by 1.28 and 1.27) (1.30)

[MATHEMATICAL EXPRESSION OMITTED] (1.31)

In particular

EjjAErr = ajrEjr (1.32)

Example 1.3

Use elementary matrices and/or unit vectors to find an expression for

(i) The product AB of the matrices A = [aij] and B = [bij].

(ii) The kth column of the product AB

(iii) The kth column of the product XYZ of the matrices X= [xij], Y = [yij] and Z = [zij]

Solutions

(i) By (1.25) and (1.29)

A = [summation]A.je'j = [summation]AEjj,

hence

[MATHEMATICAL EXPRESSION OMITTED]

(ii) (a)

(AB).k = (AB)ek = A(Bek) = AB.k by (1.16)

(b) From (i) above we can write

[MATHEMATICAL EXPRESSION OMITTED]

(iii) [MATHEMATICAL EXPRESSION OMITTED]

1.4 THE TRACE FUNCTION

The trace (or the spur) of a square matrix A of order (n X n) is the sum of the diagonal terms

[MATHEMATICAL EXPRESSION OMITTED]

We write

tr A = [summation]aii. (1.33)

From (1.19) we have

aii = e'iAei,

so that

tr A = [summation]e'iAei. (1.34)

From (1.16) and (1.34) we find

tr A = [summation]e'iA.i (1.35)

and from (1.17) and (1.34)

tr A = [summation]Ai'.e'i. (1.36)

We can obtain similar expression for the trace of a product AB of matrices.

For example

[MATHEMATICAL EXPRESSION OMITTED] (1.37)

[MATHEMATICAL EXPRESSION OMITTED] (1.38)

Similarly

[MATHEMATICAL EXPRESSION OMITTED] (1.39)

From (1.38) and (1.39) we find that

tr AB = tr BA. (1.40)

From (1.16), (1.17) and (1.37) we have

tr AB = [summation]Ai'.B.i. (1.41)

Also from (1.40) and (1.41)

tr AB = [summation]Bi.'A.i. (1.42)

Similarly

tr AB' = [summation]Ai'.Bi. (1.43)

and since tr AB' = tr A'B

tr AB' = [summation]A.'iB.i. (1.44)

Two important properties of the trace are

tr (A + B) = tr A + tr B (1.45)

and

tr (αA) = α tr A (1.46)

where α is a scalar.

These properties show that trace is a linear function.

For real matrices A and B the various properties of tr (AB') indicated above show that it is an inner product and is sometimes written as

tr (AB') = (A,B)

1.5 THE VEC OPERATOR

We shall make use of a vector valued function denoted by vec A of a matrix A defined by Neudecker [22].

If A is of order m X n

[MATHEMATICAL EXPRESSION OMITTED] (1.47)

From the definition it is clear that vec A is a vector of order mn.

For example if

[MATHEMATICAL EXPRESSION OMITTED]

then

[MATHEMATICAL EXPRESSION OMITTED]

Example 1.4

Show that we can write tr AB as (vec A')' vec B

Solution

By (1.37)

[MATHEMATICAL EXPRESSION OMITTED]

(since the ith row of A is the ith column of A')

Hence (assuming A and B of order n X n)

[MATHEMATICAL EXPRESSION OMITTED]

Before discussing a useful application of the above we must first agree on notation for the transpose of an elementary matrix, we do this with the aid of an example.

[MATHEMATICAL EXPRESSION OMITTED]

then an elementary matrix associated with will X will also be of order (2 X 3).

For example, one such matrix is

[MATHEMATICAL EXPRESSION OMITTED]

The transpose of E12 is the matrix

[MATHEMATICAL EXPRESSION OMITTED]

Although at first sight this notation for the transpose is sensible and is used frequently in this book, there are associated snags. The difficulty arises when the suffix notation is not only indicative, of the matrix involved but also determines specific elements as in equations (1.31) and (1.32). On such occasions it will be necessary to use a more accurate notation indicating the matrix order and the element involved. Then instead of E12 we will write E12 (2 X 3) and instead of E'12 we write E21 (3 X 2).

More generally if X is a matrix or order (m X n) then the transpose of

Ers (m X n)

will be written as

E'rs

unless an accurate description is necessary, in which case the transpose will be written as

Esr (n X m).

Now for the application of the result of Example 1.4 which will be used later on in the book.

From the above

tr E'rs A = (vec Ers)' (vec A) = ars

where ars is the (r,s)th element of the matrix A.

We can of course prove this important result by a more direct method.

[MATHEMATICAL EXPRESSION OMITTED]

Problems for Chapter 1

(1) The matrix A is of order (4 X n) and the matrix B is of order (n X 3). Write the product AB in terms of the rows of A, that is, A1, A2., ... and the columns of B, that is, B.1, B.2, ....

(2) Describe in words the matrices

(a) AEik and (b) EikA.

Write these matrices in terms of an appropriate product of a row or a column of A and a unit vector.

(3) Show that

(a) tr ABC = [summation over i]A'i.BC.i

(b) tr ABC = tr BCA = tr CAB

(4) Show that tr AEij = aji

(5) B = [bij] is a matrix of order (n X n) diag {B} = diag {b11, b22, ..., bnn} = [summation]biiEii.

Show that if

aij = tr BEijδij then A = [aij] = diag {B}.

CHAPTER 2

The Kronecker Product

2.1 INTRODUCTION

Kronecker product, also known as a direct product or a tensor product is a concept having its origin in group theory and has important applications in particle physics. But the technique has been successfully applied in various fields of matrix theory, for example in the solution of matrix equations which arise when using Lyapunov's approach to the stability theory. The development of the technique in this chapter will be as a topic within the scope of matrix algebra.

2.2 DEFINITION OF THE KRONECKER PRODUCT

Consider a matrix A = [aij] of order (m X n) and a matrix B = [bij] of order (r X s). The Kronecker product of the two matrices, denoted by A [??] B is defined as the partitioned matrix

[MATHEMATICAL EXPRESSION OMITTED] (2.1)

A [??] B is seen to be a matrix of order (mr X ns). It has mn blocks, the (i,j)th block is the matrix aijB of order (r X s).

For example, let

[MATHEMATICAL EXPRESSION OMITTED]

then

[MATHEMATICAL EXPRESSION OMITTED]

Notice that the Kronecker product is defined irrespective of the order of the matrices involved. From this point of view it is a more general concept than matrix multiplication. As we develop the theory we will note other results which are more general than the corresponding ones for matrix multiplication.

The Kronecker product arises naturally in the following way. Consider two Unear transformations

x = Az and y = Bw

which, in the simplest case take the form

[MATHEMATICAL EXPRESSION OMITTED] (2.2)

We can consider the two transformations simultaneously by defining the following vectors

[MATHEMATICAL EXPRESSION OMITTED] (2.3)

To find the transformation between μ and v, we determine the relations between the components of the two vectors.

For example,

[MATHEMATICAL EXPRESSION OMITTED]

Similar expressions for the other components lead to the transformation that is

[MATHEMATICAL EXPRESSION OMITTED]

or

μ = (A [??] B)(z [??] w). (2.4)

Example 2.1

Let Eij be an elementary matrix of order (2 X 2) defined in section 1.2 (see 1.4). Find the matrix

[MATHEMATICAL EXPRESSION OMITTED]

Solution

[MATHEMATICAL EXPRESSION OMITTED]

so that

[MATHEMATICAL EXPRESSION OMITTED]

Note. U is seen to be a square matrix having columns which are unit vectors ei(i = 1,2, ..). It can be obtained from a unit matrix by a permutation of rows or columns. It is known as a permutation matrix (see also section 2.5).

(Continues…)

---

Excerpted from "Matrix Calculus with Applications"
by .
Copyright © 2018 Alexander Graham.
Excerpted by permission of Dover Publications, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---

Table of Contents

Table of Contents:
Author's Preface
Symbols and Notation Used
1. Preliminaries
2. The Kronecker Product
3. Some Applications for the Kronecker Product
4. Introduction to Matrix Calculus
5. Further Development of Matrix Calculus Including an Application of Kronecker Products
6. The Derivative of a Matrix with Respect to a Matrix
7. Some Applications of Matrix Calculus
Solution to Problems
Tables of Formulae and Derivatives
Bibliography
Index