Tutorial on pre-conditioning the normal matrix

1. Tutorial on pre-conditioning the normal matrix

2. The normal matrix is the straight matrix for the weighted least squares problem. Problem: The normal matrix often has a very poor condition number Fix: Matrix pre-conditioning

3. Pre-conditioning can take several forms, with the most widespread as follows: If we can write: C = A·B·A normal matrix we then have: C-1 = [B·A]-1· A-1 = A-1· B-1· A-1 Remark: If we choose A diagonal, with Aii = (Cii)1/2 we then have Bij = Cij (Cii Cjj)-1/2 while A-1 is also diagonal with A-1ii = (Cii)-1/2 and therefore C-1 is given by C-1ij = B-1ij (Cii Cjj)-1/2 Neat algebra-101 stuff, but what is the point???

4. The point is that: • inversion of diagonal matrices gives rise to no error propagation • condition number of B is orders of magnitude lower than that of C Essentially, a problematic inversion is split into two inversions, -- one with low condition number, that can then be accurately performed, -- the other one with high condition number but no error propagation. However, there is no general mathematical proof that this splitting will work as planned. It would be risky to assume that it will work in a specific case. The proof of the pudding is then provided by SVDdiagnostic.

5. In cases where C is singular, B is also singular. SVDdiagnostic will spot the singularity on either B or C. However, the linear combination of variables that causes the singularity can be correctly read off the SVD decomposition of C only, not that of B. End of tutorial on pre-conditioning the normal matrix