Download
1 / 30
300 likes | 406 Views
Diagnostic Solvers for Linear Systems with Constraints by Rondall E. Jones Sandia National Labs, Retired www.rejonesconsulting.com rejones7@msn.com Presented by Kevin Dowding Sandia National Labs. Equation Context. We are concerned here with the general linear algebra problem:
E N D
Diagnostic Solvers for Linear Systems with ConstraintsbyRondall E. JonesSandia National Labs, Retired www.rejonesconsulting.comrejones7@msn.com Presented by Kevin DowdingSandia National Labs
Equation Context We are concerned here with the general linear algebra problem: Ax = b = b0 + e (b0 is an unknown true data vector; e is an unknown vector of errors 0 or more equations; least squares if over-determined; may be ill-conditioned and/or singular; any shape) Ex == f (0 or more equations; exact equality or else discarded) Gx >= h (0 or more equations; <= must be recast as >=)
Background • rejtrix.h C++ library (free on author’s web site) capably solves ill-conditioned linear systems with constraints, including auto-regularization based on the Discrete Picard Condition. • But complete apps were needed for some users. • We choose Excel-compatible *.csv files format. • Result was an easy-to-use suite of solvers for various weightings, manual vs auto, etc. • But… • Users wanted more information about the problem. • So more diagnostic outputs were added. • These Solve* programs are free on author’s web site.
Each solve* program uses about the same format for output. • Output varies a little with the input data and the solution method chosen • Some analyses are not possible in some circumstances • For example, some only make sense if the equations are over-determined. • Also some may only make sense if the user provides error estimates.
Sample *.csv Input File • # Will use several problems for illustrations. • # These lines starting with # are comments. • # Note use of commas in input file is optional. • # ‘=’ flag means least-squares equation; ‘!’ means ‘exact’. • # (‘<‘ and ‘>’ are for inequalities.) • # Last column is error estimates; use 0.0 if not available. • # ‘NONNEG’ shorthand adds all non-negativity constraints. • # We have 8 lines of data and 6 unknowns… • 8 6 • = 1.325,1.265, 1.21, 0.545, 1.06, 1.385 = 112.35 1.5729 • = 1.0 2.4 8.8 0.0 36.0 0.408 = 605.68 8.47952 • = 1.5 1.75 7.0 0.0 6.0 8.0 = 325.0 4.55 • = 0.4 3.0 3.7 10.0 4.4 5.2 = 410.0 5.74 • = 1.0 1.725 0.915 0.4 1.02 3.185 = 117.35 1.6429 • = 5.35 8.7 7.7 18.9 6.0 10.0 = 938.1 13.1334 • ! 1 1 1 1 1 1 = 100 0 • NONNEG Note: This mildly ill-conditioned problem is from a human enzyme profile prediction study done by Dr. Drew Jones, UAMS. All enzyme levels must be non-negative, and add to 100%.
The “advise.exe” program If you don’t know where to start, start with advise.exe. Here is the output of advise.exe for a sample problem:
An Echo of Input Problem(some columns collapsed for this display)
In Excel it takes only a couple of clicks to make a graph of the solution…
Next an analysis of the residual Note summary of constraint behavior at end. These comments will vary depending on the situation.
First the results of a Monte Carlo error analysis… To do this, we generate a number of random variations using the user’s error estimates, then discard the high and low 2.5% of each variable. We report the extremes of the remaining values. Note that the effect of ALL CONSTRAINTS is implicitly included.
An Analysis of Consistency in the Matrix and RHS Vector • Standard least squares minimizes the norm of the residual vector. • This residual vector can be viewed as the set of changes required to the RHS to make the system of equations consistent. • Total least squares can be viewed as computing the smallest set of changes to both A and b that are required to make the system consistent. • So we compute the TLS solution and display to the user the largest of the “inconsistencies” the TLS suggests.
An Example of Consistency Analysis Note: this data is from a different problem than most of the above slides, for better illustration. Note: The constraints (equality and inequality) do not play a role in this analysis.
A Different Analysis of the Matrix…(this applies only to overdetermined systems) • An old way of estimating the relative variance of the solution elements (ignoring right-hand-side issues) was to look at the diagonal elements of (ATA)-1 . • We can use the diagonal of (ATA)-1 or some function of it as a heuristic measure of model quality in a single number. (We use the square root of the average variance). • Then, we can re-measure this heuristic with an equation deleted. • If the heuristic doesn’t increase, the equation may be irrelevant. • If the heuristic increases greatly, the equation is crucial. • We call the increase in the heuristic the ‘importance’ of the equation.
Here is an Example of the “Importance” Analysis(Equation 6 may be almost useless) Note: this data is from a different problem than most of the above slides. Question: when might the “Importance” measure be negative? That is, when would the “variances” be much smaller when an equation is removed??
One Common Answer: When the system of equations is square or under-determined, and the approximate rank is one less than the row count. Then the only near-zero singular value may disappear when a particular row, or any of several rows, is deleted.
Linear Dependence Analysis (Examination of the U Matrix can reveal clear dependencies) Note: The programs will report several such sets of dependences, if present. In this case, the two rows are evidently the same (or differ by simply a scale factor.) Note: The constraints (equality and inequality) do not play a role in this analysis. Note: this data is from a different problem than most of the above slides, for better illustration.
Correlation of Solution with Rows of Matrix… Ideally the solution should be a linear combination of the rows of the matrix. It may be worthwhile to know how each row correlates to the solution. We depict this relation as an angle, using v · w = |v| |w| cos(angle between v and w).
Correlation of RHS with Columns of Matrix… Equally, the right hand side data should be a linear combination (given by the solution) of the columns of the matrix. So we depict that relationship by angles between the columns and the RHS. And there may be other such angle relationships depicted also.
Now We Give Information About the Unregularized and Unconstrained Solution by SVD (aka the Pseudo-Inverse Solution)in the next few displays…
And then the Vector S+ UT b …which is also known as the Discrete Picard Condition Vector Notice beginning of divergence toward the end of the vector.
And the (Unregularized) Pseudo-Inverse Solution x= V S+ UT b
Now, the final solution after all regularization and constraints can be multiplied by VT to show the Modification to the Picard Condition Vector… Compare this vector to the D.P.C.V. two slides back to see how the regularized and constrained solution behaves versus the pseudo-inverse solution. The latter elements behave better here.
Finally, Some Measures of Quality of the SVD Orthonormality error refers to the elements of the matrix UTU-I, for example. This matrix should be zero, within round-off.
