Finite Markov Processes and Their Applications

by Marius Iosifescu

Overview

A self-contained treatment of finite Markov chains and processes, this text covers both theory and applications. Author Marius Iosifescu, vice president of the Romanian Academy and director of its Center for Mathematical Statistics, begins with a review of relevant aspects of probability theory and linear algebra. Experienced readers may start with the second chapter, a treatment of fundamental concepts of homogeneous finite Markov chain theory that offers examples of applicable models.
The text advances to studies of two basic types of homogeneous finite Markov chains: absorbing and ergodic chains. A complete study of the general properties of homogeneous chains follows. Succeeding chapters examine the fundamental role of homogeneous infinite Markov chains in mathematical modeling employed in the fields of psychology and genetics; the basics of nonhomogeneous finite Markov chain theory; and a study of Markovian dependence in continuous time, which constitutes an elementary introduction to the study of continuous parameter stochastic processes.

Product Details

ISBN-13:	9780486458694
Publisher:	Dover Publications
Publication date:	06/05/2007
Series:	Dover Books on Mathematics Series
Pages:	304
Product dimensions:	5.37(w) x 8.50(h) x (d)

About the Author

Marius Iosifescu is vice-president of the Romanian Academy and director of its Center for Mathematical Statistics.

Table of Contents

Introduction     13
Elements of Probability Theory and Linear Algebra     17
Random events     17
Probability     20
Dependence and independence     24
Random variables. Mean values     26
Random processes     36
Matrices     38
Operations with matrices     41
r-dimensional space     46
Eigenvalues and eigenvectors     47
Nonnegative matrices. The Perron-Frobenius theorems     51
Stochastic matrices. Ergodicity coefficients     54
Fundamental Concepts in Homogeneous Markov Chain Theory     60
The Markov property     60
Examples of homogeneous Markov chains     66
Stopping times and the strong Markov property     77
Classes of states     80
Recurrence and transience     86
Classification of homogeneous Markov chains     94
Exercises     96
Absorbing Markov Chains     99
The fundamental matrix     99
Applications of the fundamental matrix     101
Extensions and complements     113
Conditional transient behaviour     116
Exercises     119
ErgodicMarkov Chains     122
Regular Markov chains     122
The stationary distribution     128
The fundamental matrix     132
Cyclic Markov chains     141
Reversed Markov chains     143
The Ehrenfest model     145
Exercises     149
General Properties of Markov Chains     153
Asymptotic behaviour of transition probabilities     153
The tail [sigma]- algebra     158
Limit theorems for partial sums     162
Grouped Markov chains     166
Expanded Markov chains     173
Extending the concept of a homogeneous finite Markov chain     175
Exercises     177
Applications of Markov Chains in Psychology and Genetics     180
Mathematical learning theory     180
The pattern model     181
The Markov chain associated with the pattern model     186
The Mendelian theory of inheritance     192
Sib mating     195
Genetic drift. The Wright model     199
Exercises     209
Nonhomogeneous Markov Chains     213
Generalities     213
Weak ergodicity     217
Uniform weak ergodicity      221
Strong ergodicity     223
Uniform strong ergodicity     226
Asymptotic behaviour of nonhomogeneous Markov chains     229
Exercises     232
Markov Processes     234
Measure theoretical definition of a Markov process     234
The intensity matrix     236
Constructive definition of a Markov process     242
Discrete skeletons and classification of states     246
Absorbing Markov processes     249
Regular Markov processes     253
Birth and death processes     257
Extending the concept of a homogeneous finite Markov process     260
Nonhomogeneous Markov processes     262
Exercises     267
Historical Notes     273
Bibliography     275
List of Symbols     291
Subject Index     293