530 likes | 715 Views
Backward Error Analysis and Numerical Software. Sven Hammarling NAG Ltd, Oxford sven@nag.co.uk. Plan of Talk. Introductio n Stability Error analysis Condition The Basic Linear Algebra Subprograms ( BLAS ) LAPACK : state of the art L inear A lgebra PACK age
E N D
Backward Error Analysis and Numerical Software Sven Hammarling NAG Ltd, Oxford sven@nag.co.uk
Plan of Talk • Introduction • Stability • Error analysis • Condition • The Basic Linear Algebra Subprograms (BLAS) • LAPACK: state of the art Linear Algebra PACKage • ScaLAPACK: distributed memory version of LAPACK (Scalable LAPACK) • Other Aspects • Summary
Stability The stability of a method for solving a problem is concerned with the sensitivity of the method to (rounding) errors in the solution process A method that guarantees as accurate a solution as the data warrants is said to be stable, otherwise the method is unstable
Error Analysis Error analysis is concerned with establishing whether or not an algorithm is stable for the problem in hand A forward error analysis is concerned with how close the computed solution is to the exact solution A backward error analysis is concerned with how well the computed solution satisfies the problem to be solved
Condition The condition of a problem is concerned with the sensitivity of the problem to perturbations in the data A problem is ill-conditioned if small changes in the data cause relatively large changes in the solution. Otherwise a problem is well-conditioned
The Purpose of Error Analysis “The clear identification of the factors determining the stability of an algorithm soon led to the development of better algorithms. The proper understanding of inverse iteration for eigenvectors and the development of the QR algorithm by Francis are the crowning achievements of this line of research. “For me, then, the primary purpose of the rounding error analysis was insight.” Wilkinson, 1986 Bulletin of the IMA, Vol 22, p197
LAPACK and Error Bounds “In Addition to providing faster routines than previously available, LAPACK provides more comprehensive and better error bounds. Our goal is to provide error bounds for most quantities computed by LAPACK.” LAPACK Users’ Guide, Chapter 4 – Accuracy and Stability
The Level 1 BLAS are concerned with scalar and vector operations, such as the Level 2 BLAS with matrix-vector operations such as and the Level 3 BLAS with matrix-matrix operations such as The Three Levels of BLAS
BLAS Mem Refs Ops Ratio Level 1 3n 2n 3 : 2 Level 2 n2 2n2 1 : 2 Level 3 Registers 4n2 2n3 2 : n L1 Cache L2 Cache Local Memory Remote Memory Secondary Memory Why Higher Level BLAS? The so called ‘surface to volume’ effect
BLAS for Performance Level 3 BLAS Level 2 BLAS Level 1 BLAS
Publication of the Specifications of the BLAS [1] C L. Lawson, R. J. Hanson, D. Kincaid, and F. T. Krogh. Basic Linear Algebra Subprograms for FORTRAN usage. ACM Trans. Math. Software, 5:308-323, 1979. [2] J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson. An extended set of FORTRAN Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 14:1-32, 399, 1988. (Algorithm 656). [3] J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling. A set of Level 3 Basic Linear Algebra Subprograms. ACM Trans. Math. Software, 16:1-28, 1990. (Algorithm 679).
LAPACK Linear Algebra PACKage for high-performance computers • Systems of linear equations • Linear least squares problems • Eigenvalue and singular value problems, including generalized problems • Matrix factorizations • Condition and error estimates • The BLAS as a portability layer Dense and banded linear algebra for Shared Memory
Benefits of LAPACK • Comprehensive and consistent set of routines for dense and banded linear algebra • Efficient on a wide range of modern computers • Additional functionality compared to previous packages • Improved numerical algorithms • Error and sensitivity estimates • Portability layer between users applications and the machines
L AP A C K L -AP -A C -K L AP A-C -K L -AP -A-C K L A -P -AC K L -A -P AC -K LAPACK Users’ Guide E. Anderson, Z. Bai, C. Bischof S. Blackford, J. Demmel, J. Dongarra J. Du Croz, A. Greenbaum, S. Hammarling A. McKenney, D. Sorensen SIAM, 1999 (3rd Edition) Proceeds to SIAM student travel fund http://www.netlib.org/lapack/lug/lapack_lug.html
1/L 1/L 1/L 1/L 1/A -1/A 1/A -1/A 1/P 1/P -1/P -1/P 1/A -1/A -1/A 1/A 1/C 1/C -1/C -1/C 1/K -1/K -1/K 1/K 1/4
Performance Improvements • Efficiency in LAPACK has been principally achieved by restructuring algorithms so that as much computation as possible is performed by calls to: Level 3 and Level 2 BLAS • In order to exploit the Level 3 BLAS we needed: BLOCK-PARTITIONED ALGORITHMS a11 a12 a13 a21a22a23 a31 a32a33 A11 A12 A13 A21 A22 A23 A31 A32 A33
Error Bounds in LAPACK The Users’ Guide gives details of error bounds and code fragments to compute those bounds. In many cases the routines return the bounds directly
Error Bounds in LAPACK: Example 2 (cont’d) Asymptotic error bounds for nonsymmetric eigenproblem See Golub and Van Loan – Matrix Computations
Error Bounds in LAPACK: Example 2 (cont’d) EPSMCH = DLAMCH(‘E’) DO 10 I = 1, N EERRBD(I) = EPSMCH*ABNRM/RCONDE(I) VERRBD(I) = EPSMCH*ABNRM/RCONDV(I) 10 CONTINUE EPSMCH is the machine precision
Availability of LAPACK LAPACKis freely available vianetlib: http://www.netlib.org/lapack/index.html Much ofLAPACKis included in the NAG Fortran 77 Library and is the basis of the dense linear algebra in the NAG Fortran 90 Library Tuned versions of someLAPACKroutines are in the NAG Fortran SMP Library
LAPACK - Summary • LAPACK success from HP-48G to Cray T-90 • Portable Fortran 77 (+BLAS), 766K lines (354K library, 412K testing and timing) • PCs, workstations, vector machines, and shared memory computers • Initial release February 29th, 1992 • Now on the 3rd release • Linear systems, least squares, eigenproblems • C, Java and Fortran 95 interfaces
ScaLAPACK Scalable Linear Algebra PACKage Aim: To portLAPACK to distributed memory environments • Scalability, performance and portability • Interfaces to stay as close to LAPACK as possible • Use LAPACKalgorithms wherever possible • Principal additional consideration is the data distribution • The PBLAS (BLAS + BLACS) as a portabilility layer As withLAPACK, ScaLAPACKis freely available: http://www.netlib.org/scalapack/index.html Most ofScaLAPACKincluded in NAG Parallel Library
ScaLAPACK Structure ScaLAPACK PBLAS Global Local LAPACK BLACS BLAS PVM/MPI/...
ScaLAPACK Summary • Follow on to LAPACK - Designed for MPP’s • Object based design • Portability across wide range of architectures • will work on a heterogeneous platform • Software, examples, prebuilt libraries available fromnetlib • Users’ Guide, including CD available from SIAM • Continuing development • HPF interface to some driver routines available
Hierarchy of Building Blocks Users Applications Packages (Sca)LAPACK (P)BLAS Machines
Developments Since Wilkinson • More problems solved, tighter bounds • More emphasis on componentwise results – particularly for sparse and structured problems • Use of standard precision iterative refinement • Importance of scaling • Iterative methods • Importance of non-normality and pseudospectra
Backward Errors in Other Areas • For references to other areas see: Higham, Accuracy and Stability of Numerical Algorithms, Sections 1.20 & 1.21. • Examples: • ODEs • Chaotic behaviour of iterations
What to do if Software does not Provide Estimates? • Run the problem with perturbed data • Better still, use a software tool such as PRECISE which allows you to perform a stochastic analysis • Put pressure on developers to provide estimates
PRECISE • Provides a module for statistical backward error analysis • Provides a module for sensitivity analysis • http://www.cerfacs.fr/algor/Softs/PRECISE/index.html