Numerical Methods: Design, Analysis, and Computer Implementation of Algorithms

by Anne Greenbaum, Tim P. Chartier

Overview

A rigorous and comprehensive introduction to numerical analysis

Numerical Methods provides a clear and concise exploration of standard numerical analysis topics, as well as nontraditional ones, including mathematical modeling, Monte Carlo methods, Markov chains, and fractals. Filled with appealing examples that will motivate students, the textbook considers modern application areas, such as information retrieval and animation, and classical topics from physics and engineering. Exercises use MATLAB and promote understanding of computational results.

The book gives instructors the flexibility to emphasize different aspects—design, analysis, or computer implementation—of numerical algorithms, depending on the background and interests of students. Designed for upper-division undergraduates in mathematics or computer science classes, the textbook assumes that students have prior knowledge of linear algebra and calculus, although these topics are reviewed in the text. Short discussions of the history of numerical methods are interspersed throughout the chapters. The book also includes polynomial interpolation at Chebyshev points, use of the MATLAB package Chebfun, and a section on the fast Fourier transform. Supplementary materials are available online.

• Clear and concise exposition of standard numerical analysis topics
• Explores nontraditional topics, such as mathematical modeling and Monte Carlo methods
• Covers modern applications, including information retrieval and animation, and classical applications from physics and engineering
• Promotes understanding of computational results through MATLAB exercises
• Provides flexibility so instructors can emphasize mathematical or applied/computational aspects of numerical methods or a combination
• Includes recent results on polynomial interpolation at Chebyshev points and use of the MATLAB package Chebfun
• Short discussions of the history of numerical methods interspersed throughout
• Supplementary materials available online

Product Details

ISBN-13:	9780691151229
Publisher:	Princeton University Press
Publication date:	04/01/2012
Edition description:	New Edition
Pages:	464
Sales rank:	627,551
Product dimensions:	7.20(w) x 10.10(h) x 1.20(d)

About the Author

Anne Greenbaum is professor of applied mathematics at the University of Washington. She is the author of Iterative Methods for Solving Linear Systems. Timothy P. Chartier is associate professor of mathematics at Davidson College.

Table of Contents

Preface xiii

Chapter 1: MATHEMATICAL MODELING 1
1.1 Modeling in Computer Animation 2
1.1.1 A Model Robe 2
1.2 Modeling in Physics: Radiation Transport 4
1.3 Modeling in Sports 6
1.4 Ecological Models 8
1.5 Modeling a Web Surfer and Google 11
1.5.1 The Vector Space Model 11
1.5.2 Google’s PageRank 13
1.6 Chapter 1 Exercises 14

Chapter 2: BASIC OPERATIONS WITH MATLAB 19
2.1 Launching MATLAB 19
2.2 Vectors 20
2.3 Getting Help 22
2.4 Matrices 23
2.5 Creating and Running .m Files 24
2.6 Comments 25
2.7 Plotting 25
2.8 Creating Your Own Functions 27
2.9 Printing 28
2.10 More Loops and Conditionals 29
2.11 Clearing Variables 31
2.12 Logging Your Session 31
2.13 More Advanced Commands 31
2.14 Chapter 2 Exercises 32

Chapter 3: MONTE CARLO METHODS 41
3.1 A Mathematical Game of Cards 41
3.1.1 The Odds in Texas Holdem 42
3.2 Basic Statistics 46
3.2.1 Discrete Random Variables 48
3.2.2 Continuous Random Variables 51
3.2.3 The Central Limit Theorem 53
3.3 Monte Carlo Integration 56
3.3.1 Buffon’s Needle 56
3.3.2 Estimating π 58
3.3.3 Another Example of Monte Carlo Integration 60
3.4 Monte Carlo Simulation of Web Surfing 64
3.5 Chapter 3 Exercises 67

Chapter 4: SOLUTION OF A SINGLE NONLINEAR EQUATION IN ONE UNKNOWN 71
4.1 Bisection 75
4.2 Taylor’s Theorem 80
4.3 Newton’s Method 83
4.4 Quasi-Newton Methods 89
4.4.1 Avoiding Derivatives 89
4.4.2 Constant Slope Method 89
4.4.3 Secant Method 90
4.5 Analysis of Fixed Point Methods 93
4.6 Fractals, Julia Sets, and Mandelbrot Sets 98
4.7 Chapter 4 Exercises 102

Chapter 5: FLOATING-POINT ARITHMETIC 107
5.1 Costly Disasters Caused by Rounding Errors 108
5.2 Binary Representation and Base 2 Arithmetic 110
5.3 Floating-Point Representation 112
5.4 IEEE Floating-Point Arithmetic 114
5.5 Rounding 116
5.6 Correctly Rounded Floating-Point Operations 118
5.7 Exceptions 119
5.8 Chapter 5 Exercises 120

Chapter 6: CONDITIONING OF PROBLEMS; STABILITY OF ALGORITHMS 124
6.1 Conditioning of Problems 125
6.2 Stability of Algorithms 126
6.3 Chapter 6 Exercises 129

Chapter 7: DIRECT METHODS FOR SOLVING LINEAR SYSTEMS AND LEAST SQUARES PROBLEMS 131
7.1 Review of Matrix Multiplication 132
7.2 Gaussian Elimination 133
7.2.1 Operation Counts 137
7.2.2 LU Factorization 139
7.2.3 Pivoting 141
7.2.4 Banded Matrices and Matrices for Which Pivoting Is Not Required 144
7.2.5 Implementation Considerations for High Performance 148
7.3 Other Methods for Solving Ax = b 151
7.4 Conditioning of Linear Systems 154
7.4.1 Norms 154
7.4.2 Sensitivity of Solutions of Linear Systems 158
7.5 Stability of Gaussian Elimination with Partial Pivoting 164
7.6 Least Squares Problems 166
7.6.1 The Normal Equations 167
7.6.2 QR Decomposition 168
7.6.3 Fitting Polynomials to Data 171
7.7 Chapter 7 Exercises 175

Chapter 8: POLYNOMIAL AND PIECEWISE POLYNOMIAL INTERPOLATION 181
8.1 The Vandermonde System 181
8.2 The Lagrange Form of the Interpolation Polynomial 181
8.3 The Newton Form of the Interpolation Polynomial 185
8.3.1 Divided Differences 187
8.4 The Error in Polynomial Interpolation 190
8.5 Interpolation at Chebyshev Points and chebfun 192
8.6 Piecewise Polynomial Interpolation 197
8.6.1 Piecewise Cubic Hermite Interpolation 200
8.6.2 Cubic Spline Interpolation 201
8.7 Some Applications 204
8.8 Chapter 8 Exercises 206

Chapter 9: NUMERICAL DIFFERENTIATION AND RICHARDSON EXTRAPOLATION 212
9.1 Numerical Differentiation 213
9.2 Richardson Extrapolation 221
9.3 Chapter 9 Exercises 225

Chapter 10: NUMERICAL INTEGRATION 227
10.1 Newton-Cotes Formulas 227
10.2 Formulas Based on Piecewise Polynomial Interpolation 232
10.3 Gauss Quadrature 234
10.3.1 Orthogonal Polynomials 236
10.4 Clenshaw-Curtis Quadrature 240
10.5 Romberg Integration 242
10.6 Periodic Functions and the Euler-Maclaurin Formula 243
10.7 Singularities 247
10.8 Chapter 10 Exercises 248

Chapter 11: NUMERICAL SOLUTION OF THE INITIAL VALUE PROBLEM FOR ORDINARY DIFFERENTIAL EQUATIONS 251
11.1 Existence and Uniqueness of Solutions 253
11.2 One-Step Methods 257
11.2.1 Euler’s Method 257
11.2.2 Higher-Order Methods Based on Taylor Series 262
11.2.3 Midpoint Method 262
11.2.4 Methods Based on Quadrature Formulas 264
11.2.5 Classical Fourth-Order Runge-Kutta and Runge-Kutta-Fehlberg Methods 265
11.2.6 An Example Using MATLAB’s ODE Solver 267
11.2.7 Analysis of One-Step Methods 270
11.2.8 Practical Implementation Considerations 272
11.2.9 Systems of Equations 274
11.3 Multistep Methods 275
11.3.1 Adams-Bashforth and Adams-Moulton Methods 275
11.3.2 General Linear m-Step Methods 277
11.3.3 Linear Difference Equations 280
11.3.4 The Dahlquist Equivalence Theorem 283
11.4 Stiff Equations 284
11.4.1 Absolute Stability 285
11.4.2 Backward Differentiation Formulas (BDF Methods) 289
11.4.3 Implicit Runge-Kutta (IRK) Methods 290
11.5 Solving Systems of Nonlinear Equations in Implicit Methods 291
11.5.1 Fixed Point Iteration 292
11.5.2 Newton’s Method 293
11.6 Chapter 11 Exercises 295

Chapter 12: MORE NUMERICAL LINEAR ALGEBRA: EIGENVALUES AND ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS 300
12.1 Eigenvalue Problems 300
12.1.1 The Power Method for Computing the Largest Eigenpair 310
12.1.2 Inverse Iteration 313
12.1.3 Rayleigh Quotient Iteration 315
12.1.4 The QR Algorithm 316
12.1.5 Google’s PageRank 320
12.2 Iterative Methods for Solving Linear Systems 327
12.2.1 Basic Iterative Methods for Solving Linear Systems 327
12.2.2 Simple Iteration 328
12.2.3 Analysis of Convergence 332
12.2.4 The Conjugate Gradient Algorithm 336
12.2.5 Methods for Nonsymmetric Linear Systems 334
12.3 Chapter 12 Exercises 345

Chapter 13: NUMERICAL SOLUTION OF TWO-POINT BOUNDARY VALUE PROBLEMS 350
13.1 An Application: Steady-State Temperature Distribution 350
13.2 Finite Difference Methods 352
13.2.1 Accuracy 354
13.2.2 More General Equations and Boundary Conditions 360
13.3 Finite Element Methods 365
13.3.1 Accuracy 372
13.4 Spectral Methods 374
13.5 Chapter 13 Exercises 376

Chapter 14: NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 379
14.1 Elliptic Equations 381
14.1.1 Finite Difference Methods 381
14.1.2 Finite Element Methods 386
14.2 Parabolic Equations 388
14.2.1 Semidiscretization and the Method of Lines 389
14.2.2 Discretization in Time 389
14.3 Separation of Variables 396
14.3.1 Separation of Variables for Difference Equations 400
14.4 Hyperbolic Equations 402
14.4.1 Characteristics 402
14.4.2 Systems of Hyperbolic Equations 403
14.4.3 Boundary Conditions 404
14.4.4 Finite Difference Methods 404
14.5 Fast Methods for Poisson’s Equation 409
14.5.1 The Fast Fourier Transform 411
14.6 Multigrid Methods 414
14.7 Chapter 14 Exercises 418

APPENDIX A REVIEW OF LINEAR ALGEBRA 421
A.1 Vectors and Vector Spaces 421
A.2 Linear Independence and Dependence 422
A.3 Span of a Set of Vectors; Bases and Coordinates; Dimension of a Vector Space 423
A.4 The Dot Product; Orthogonal and Orthonormal Sets; the Gram-Schmidt Algorithm 423
A.5 Matrices and Linear Equations 425
A.6 Existence and Uniqueness of Solutions; the Inverse; Conditions for Invertibility 427
A.7 Linear Transformations; the Matrix of a Linear Transformation 431
A.8 Similarity Transformations; Eigenvalues and Eigenvectors 432
APPENDIX B TAYLOR’S THEOREM IN MULTIDIMENSIONS 436

References 439
Index 445

What People are Saying About This

From the Publisher

"This is an excellent introduction to the exciting world of numerical analysis. Fulfilling the need for a modern textbook on numerical methods, this volume has a wealth of examples that will keep students interested in the material. The mathematics is completely rigorous and I applaud the authors for doing such a marvelous job."—Michele Benzi, Emory University

"Filled with polished details and a plethora of examples and illustrations, this ambitious and substantial text touches every standard topic of numerical analysis. The authors have done a huge amount of work and produced a major textbook for this subject."—Lloyd N. Trefethen, University of Oxford

Editorial Reviews

"Distinguishing features are the inclusion of many recent applications of numerical methods and the extensive discussion of methods based on Chebyshev interpolation. This book would be suitable for use in courses aimed at advanced undergraduate students in mathematics, the sciences, and engineering."

Choice

"[Numerical Methods] is a very pleasant book, where the concepts involved are clearly explained. All chapters begin with motivating examples that give a precise idea of the methods developed. In addition, every chapter ends with an extensive collection of exercises, useful to understand the importance of the results. These are complemented by a series of exercises, designed to be performed with Matlab, useful to appreciate the behavior of the methods studied."

European Mathematical Society

"An instructor could assemble several different one-semester courses using this book—numerical linear algebra and interpolation, or numerical solutions of differential equations—or perhaps a two-semester sequence. This is a charming book, well worth consideration for the next numerical analysis course."—-William J. Satzer, MAA Focus

From the Publisher

An instructor could assemble several different one-semester courses using this book—numerical linear algebra and interpolation, or numerical solutions of differential equations—or perhaps a two-semester sequence. This is a charming book, well worth consideration for the next numerical analysis course.

MAA Focus - William J. Satzer

Despite the ordinariness of the topics, there is a pleasant freshness here. The writing is lively, the visual presentation is appealing, and there are some great exercises.
— William J. Satzer

MAA Focus