Iterative Solvers for Linear Systems of Equations

Iterative Solvers for Linear Systems of Equations • Presented by: • Kaveh Rahnema • rahnema@in.tum.de • Supervisor: • Dr. Stefan Zimmer • zimmer@in.tum.de

Outline • Short explanation of the system • Jacobi method • Eigenvalue analysis of the Jacobi method • The Multigrid Method

Short explanation of the system To solve this equation numerically we need to descretize it and bring it into finite difference notation

Jacobi method The system can be written as Where D is the Diagonal , L is the lower triangle and U is the upper triangle part of the matrix A If we define the Jacobi iteration matrix as Then the Jacobi method appears in matrix form as

Modified Jacobi The Jacobi method can be modified as Where w is a damping factor to help Jacobi converge faster to the solution Choosing w = 1 we get the original Jacobi method Now that we have the formulation of the Jacobi how should we use it to solve the system? We start with an initial guess (sometimes might be even a bad guess) and solve the system iteratively using the results from the last iteration to perform the current step until our stopping criteria is satisfied For stopping criteria we can use different schemes, one might use the difference between 2 iterations , residual etc.

Eigenvalue analysis of the Jacobi method The Jacobi formulation can also be written as And we have It follows that the eigenvalues of Pw and A are related by

Eigenvalue analysis of the Jacobi method Now the problem becomes finding the eigenvalues of A And A and P have the same eigenvectors We can expand the error of the initial guess using the eigenvalues of A

Eigenvalue analysis of the Jacobi method The error after n iterations is given by As we see after n iterations the error is only reduced by a factor but the fourier modes don‘t mix and the condition for Modified Jacobi to converge is In the next slide you see the plot of damping factor of Pwfor fourier modes for 4 different w

Frequency dependent damping

Example solved by Jacobi method Here as an example we solve the equation In the [0,1] domain with boundary condition To solve this system we started with a random initial guess, which might not be used often but for some reasons is a good guess for this presentation As you will see in the next slide the Jacobi method works good to damp the high frequencies but damps the low frequencies slow, This makes solving systems of equations with Jacobi hard, and one may look for other better solvers or techniques to solve the equations faster

Modified Jacobi Damping Clip 100 90 80 70 60 50 40 30 20 10 0 Value at Grid Points SLOW 0 20 40 60 80 100 120 Grid Points

Example solved by Jacobi method As said before the high frequencies damp fast but the low frequencies remain and damp slow This gets even worse going to finer grid points What shall we do?! Shall we wait for a long time till we get the result when we have a large number of grid points?!!

The Multigrid Method As we saw in the last slides the Jacobi method is slow for the low frequencies But let‘s take look at the initial guess at the coarser grid Let‘s take every second point for the next coarser grid The next coarse grid has half as much grid points as the fine grid has, so the fourier modes represented in the fine grid can be rewritten as follows for the coarse grid This means that the kth mode on Ωh becomes the kth mode on Ω2h, which means that in passing from the fine grid to the coarse grid, a mode becomes more oscillatory

The Multigrid Method Now that we know the smooth modes in the fine grid are oscilatory in the next coarse grids, what shall we do? How about doing some iterations of the Jacobi in the fine grid to get a smoother solution and then pass the residual every second point to the coarser grid and do more smoothing there? What shall we do when we are done with the smoothing on the coarse grid? Shall we return to the fine grid? How? We don‘t have the same dimension! Shall we go to the next coarser grid? Why? We continue going down to the next coarser grid till we are in the coarsetst grid where we only have the lowest frequency mode left And then we need to find a way to go back to the finer grids and check our stopping criteria

Going back to the fine grid The next step is to go from the coarsest grid one level up to the next fine grid, But for this step we need to generate the intermediate points which we lost while going to corase grid The interpolation techniques might help us through this We can regenerate the intermediate points by interpolating them using the 2 neighboring points Now when we are one level finner, we will do some more iterations of the Jacobi method to avoid the high frequency oscillations occured from going to coarse grid and coming back and continue going up using the same steps till we are in the finest grid

Back in the finest grid Now that we are in the finest (original) grid what shall we do? Are we done with solving the system? Is the solution good enough? Now we will check our resedual and if the resedual is what we expect we stop the solver and we have the solution, else we have to redo the same operation till we reach our expected residual One multigrid cycle is also called a V cycle since we go down and come back up

Multigrid code • Pre-smoothing: Perform some iterations of Modified Jacobi to get a rather smooth solution • Construct the residue • Restrict your residue to every second point to go to the coarse grid • Do step 1 • Goto step 2 till you are in the coarsest grid • Interpolate your results and go back to the finer grid • Add the result from the coarse grid to the previous results • Post-smoothing: perform some iteration of Modified Jacobi • Goto step 5 till you are in the finest grid • Check the residual • If condition not satisfied start from 1 • else return the solution

The expected residual achieved!!

References • Prof. Bungartz, Numerical Programming course, TUM (CSE), WS 2007 • Dr. Bader, Scientific Computing Course, TUM (CSE), WS 2007 • Dr. Mehl, Scientific Computing Lab Course, TUM (CSE), WS 2007 • William L. Briggs, A Multigirid Tutorial (1987)

Thank you for attending this talk

Iterative Solvers for Linear Systems of Equations