1 / 53

Backward Error Analysis and Numerical Software

Backward Error Analysis and Numerical Software. Sven Hammarling NAG Ltd, Oxford sven@nag.co.uk. Plan of Talk. Introductio n Stability Error analysis Condition The Basic Linear Algebra Subprograms ( BLAS ) LAPACK : state of the art L inear A lgebra PACK age

kelda
Download Presentation

Backward Error Analysis and Numerical Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Backward Error Analysis and Numerical Software Sven Hammarling NAG Ltd, Oxford sven@nag.co.uk

  2. Plan of Talk • Introduction • Stability • Error analysis • Condition • The Basic Linear Algebra Subprograms (BLAS) • LAPACK: state of the art Linear Algebra PACKage • ScaLAPACK: distributed memory version of LAPACK (Scalable LAPACK) • Other Aspects • Summary

  3. Stability The stability of a method for solving a problem is concerned with the sensitivity of the method to (rounding) errors in the solution process A method that guarantees as accurate a solution as the data warrants is said to be stable, otherwise the method is unstable

  4. Example - Sample Variance

  5. Error Analysis Error analysis is concerned with establishing whether or not an algorithm is stable for the problem in hand A forward error analysis is concerned with how close the computed solution is to the exact solution A backward error analysis is concerned with how well the computed solution satisfies the problem to be solved

  6. Example

  7. Example (cont’d)

  8. Componentwise and Normwise

  9. Condition The condition of a problem is concerned with the sensitivity of the problem to perturbations in the data A problem is ill-conditioned if small changes in the data cause relatively large changes in the solution. Otherwise a problem is well-conditioned

  10. Example

  11. Condition Number for Linear Equations

  12. Condition and Error Analysis

  13. Example

  14. Backward Error as Perturbations in the Data

  15. Backward Error as … (cont’d)

  16. Example

  17. Backward Error Analysis

  18. Backward Error and Perturbation Analysis

  19. The Purpose of Error Analysis “The clear identification of the factors determining the stability of an algorithm soon led to the development of better algorithms. The proper understanding of inverse iteration for eigenvectors and the development of the QR algorithm by Francis are the crowning achievements of this line of research. “For me, then, the primary purpose of the rounding error analysis was insight.” Wilkinson, 1986 Bulletin of the IMA, Vol 22, p197

  20. LAPACK and Error Bounds “In Addition to providing faster routines than previously available, LAPACK provides more comprehensive and better error bounds. Our goal is to provide error bounds for most quantities computed by LAPACK.” LAPACK Users’ Guide, Chapter 4 – Accuracy and Stability

  21. The BLAS, LAPACK and ScaLAPACK

  22. The Level 1 BLAS are concerned with scalar and vector operations, such as the Level 2 BLAS with matrix-vector operations such as and the Level 3 BLAS with matrix-matrix operations such as The Three Levels of BLAS

  23. BLAS Mem Refs Ops Ratio Level 1 3n 2n 3 : 2 Level 2 n2 2n2 1 : 2 Level 3 Registers 4n2 2n3 2 : n L1 Cache L2 Cache Local Memory Remote Memory Secondary Memory Why Higher Level BLAS? The so called ‘surface to volume’ effect

  24. BLAS for Performance Level 3 BLAS Level 2 BLAS Level 1 BLAS

  25. Publication of the Specifications of the BLAS [1] C L. Lawson, R. J. Hanson, D. Kincaid, and F. T. Krogh. Basic Linear Algebra Subprograms for FORTRAN usage. ACM Trans. Math. Software, 5:308-323, 1979. [2] J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson. An extended set of FORTRAN Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 14:1-32, 399, 1988. (Algorithm 656). [3] J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling. A set of Level 3 Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 16:1-28, 1990. (Algorithm 679).

  26. LAPACK Linear Algebra PACKage for high-performance computers • Systems of linear equations • Linear least squares problems • Eigenvalue and singular value problems, including generalized problems • Matrix factorizations • Condition and error estimates • The BLAS as a portability layer Dense and banded linear algebra for Shared Memory

  27. Benefits of LAPACK • Comprehensive and consistent set of routines for dense and banded linear algebra • Efficient on a wide range of modern computers • Additional functionality compared to previous packages • Improved numerical algorithms • Error and sensitivity estimates • Portability layer between users applications and the machines

  28. L AP A C K L -AP -A C -K L AP A-C -K L -AP -A-C K L A -P -AC K L -A -P AC -K LAPACK Users’ Guide E. Anderson, Z. Bai, C. Bischof S. Blackford, J. Demmel, J. Dongarra J. Du Croz, A. Greenbaum, S. Hammarling A. McKenney, D. Sorensen SIAM, 1999 (3rd Edition) Proceeds to SIAM student travel fund http://www.netlib.org/lapack/lug/lapack_lug.html

  29. 1/L 1/L 1/L 1/L 1/A -1/A 1/A -1/A 1/P 1/P -1/P -1/P 1/A -1/A -1/A 1/A 1/C 1/C -1/C -1/C 1/K -1/K -1/K 1/K 1/4

  30. Performance Improvements • Efficiency in LAPACK has been principally achieved by restructuring algorithms so that as much computation as possible is performed by calls to: Level 3 and Level 2 BLAS • In order to exploit the Level 3 BLAS we needed: BLOCK-PARTITIONED ALGORITHMS a11 a12 a13 a21a22a23 a31 a32a33 A11 A12 A13 A21 A22 A23 A31 A32 A33

  31. Error Bounds in LAPACK The Users’ Guide gives details of error bounds and code fragments to compute those bounds. In many cases the routines return the bounds directly

  32. Error Bounds in LAPACK: Example 1

  33. Error Bounds in LAPACK: Example 2

  34. Error Bounds in LAPACK: Example 2 (cont’d) Asymptotic error bounds for nonsymmetric eigenproblem See Golub and Van Loan – Matrix Computations

  35. Error Bounds in LAPACK: Example 2 (cont’d) EPSMCH = DLAMCH(‘E’) DO 10 I = 1, N EERRBD(I) = EPSMCH*ABNRM/RCONDE(I) VERRBD(I) = EPSMCH*ABNRM/RCONDV(I) 10 CONTINUE EPSMCH is the machine precision

  36. Availability of LAPACK LAPACKis freely available vianetlib: http://www.netlib.org/lapack/index.html Much ofLAPACKis included in the NAG Fortran 77 Library and is the basis of the dense linear algebra in the NAG Fortran 90 Library Tuned versions of someLAPACKroutines are in the NAG Fortran SMP Library

  37. LAPACK - Summary • LAPACK success from HP-48G to Cray T-90 • Portable Fortran 77 (+BLAS), 766K lines (354K library, 412K testing and timing) • PCs, workstations, vector machines, and shared memory computers • Initial release February 29th, 1992 • Now on the 3rd release • Linear systems, least squares, eigenproblems • C, Java and Fortran 95 interfaces

  38. ScaLAPACK Scalable Linear Algebra PACKage Aim: To portLAPACK to distributed memory environments • Scalability, performance and portability • Interfaces to stay as close to LAPACK as possible • Use LAPACKalgorithms wherever possible • Principal additional consideration is the data distribution • The PBLAS (BLAS + BLACS) as a portabilility layer As withLAPACK, ScaLAPACKis freely available: http://www.netlib.org/scalapack/index.html Most ofScaLAPACKincluded in NAG Parallel Library

  39. ScaLAPACK Structure ScaLAPACK PBLAS Global Local LAPACK BLACS BLAS PVM/MPI/...

  40. ScaLAPACK Summary • Follow on to LAPACK - Designed for MPP’s • Object based design • Portability across wide range of architectures • will work on a heterogeneous platform • Software, examples, prebuilt libraries available fromnetlib • Users’ Guide, including CD available from SIAM • Continuing development • HPF interface to some driver routines available

  41. Hierarchy of Building Blocks Users Applications Packages (Sca)LAPACK (P)BLAS Machines

  42. Developments Since Wilkinson • More problems solved, tighter bounds • More emphasis on componentwise results – particularly for sparse and structured problems • Use of standard precision iterative refinement • Importance of scaling • Iterative methods • Importance of non-normality and pseudospectra

  43. Diagonal Matrix Example

  44. Backward Errors in Other Areas • For references to other areas see: Higham, Accuracy and Stability of Numerical Algorithms, Sections 1.20 & 1.21. • Examples: • ODEs • Chaotic behaviour of iterations

  45. What to do if Software does not Provide Estimates? • Run the problem with perturbed data • Better still, use a software tool such as PRECISE which allows you to perform a stochastic analysis • Put pressure on developers to provide estimates

  46. PRECISE • Provides a module for statistical backward error analysis • Provides a module for sensitivity analysis • http://www.cerfacs.fr/algor/Softs/PRECISE/index.html

More Related