Flexible Local Approximation Schemes and Parallel Generalized Finite Element Method for Multiparticle Problems

1. Flexible Local Approximation Schemes and Parallel Generalized Finite Element Method for Multiparticle Problems Achim Basermann NEC Europe Ltd., C&C Research Laboratories, Rathausallee 10, D-53757 Sankt Augustin, Germany basermann@ccrl-nece.de Igor Tsukerman Department of Electrical and Computer Engineering, The University of Akron, OH 44325-3904 igor@uakron.edu

2. Outline • Motivation: application examples. • Flexible Approximation Methods: Generalized FEM, Variational-Difference schemes, FD-Trefftz. Simple grids for complex problems. • Parallel implementation of GFEM and numerical results. • Future work and applications. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

3. Collaborators • NSF-NIRT and other projects. Prof. Gary Friedman’s group at Drexel University, Philadelphia. • NSF-NIRT project. Prof. Sanford Asher, Chemistry, the University of Pittsburgh. Sara Majetich, Physics, Carnegie Mellon. • Dr. Christian Holm, Polyelectrolyte group, MPI-Mainz (Max-Planck Institut für Polymerforschung), Germany. The Espresso Package (Extensible Simulation Package for Research on Soft matter). Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

4. Motivation: application examples Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

5. MAGDA: MAGnetically Driven Assembly (G. Friedman) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

6. Why Magnetically Driven Assembly? • Magnetic forces have a much longer reach than surface forces and forces of chemical nature. • Unlike electrostatic fields, magnetic fields of colloidal particles are not screened by ions in solutions and are largely independent of the fluid being employed. • Magnetic fields do not interfere with the biological, chemical or electronic functions of biomolecules, inorganic molecules and/or live cells. • Both repulsive and attractive effects possible due to nonlinearity of ferromagnetic materials. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

7. Selective printing of colloidal particles (G. Friedman) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

8. Experiments with an External Bias Field (G. Friedman) 2.8 micron Dynal beads were used Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

9. Colloidal Particles Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

10. Guiding Principle for New Schemes:Freedom of Approximation Waves, boundary layers, singularities, field jumps, dipolar potential (a schematic illustration). Different local approximations of the field in different regions are desirable. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

11. Standard Methods – Generic Approximation • FD Schemes: Taylor expansion. • FEM: piecewise-polynomial approximation (adaptivity possible). • Generalized FEM: much higher flexibility of approximation, but at a cost. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

12. First Mini-Summary: “Special is Typical” • Special behavior of the field is not special! It needs to be taken into account in various applications. • Standard computational methods, as a rule, use generic approximations of the solution and do not take full advantage of a priori knowledge of its behavior. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

13. Flexible Approximation Methods • Generalized FEM. • Variational-Difference schemes. • Finite Difference – Trefftz schemes. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

14. 2 1 3 4 Generalized FEM by Partition of Unity: Overlapping Patches The domain is covered by arbitrary overlapping patches i Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

15. φ1 φ2 Ω1 Ω2 Partition of Unity (used in the Generalized FEM) approximation error: Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

16. Partition of Unity (continued) Hence any function can be decomposed into its “patch components” The same is true for any approximating function: … and for the approximation error: Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

17. Ψ = { 1, , }  n = n Examples of Special Approximating Functions at Material Boundaries 1. Spatial mapping. Babuška I, Caloz G., Osborn J.E.:, SIAM Journal on Numerical Analysis, 31, No. 4 (1994), 945-981. 2. Spherical harmonics. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

18. Disadvantages of the Generalized FEM • Multiplication by the “partition of unity” functions makes the approximating functions more complicated. • Possible ill-conditioning or even linear dependence of the approximating functions. • Numerical quadratures in geometrically complex 3D regions. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

19. A New Framework: Multivalued Approximation • Use any desired approximating functions within each patch. • When patches overlap, the approximation is generally multivalued. • Use simple regular grids. • “Information transfer” between patches through (unique) nodal values on the grid. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

20. New Variational-Difference Schemes: Simple Grids for Complex Problems • Incorporating any desired analytical approximation of the potential (e.g. discontinuities) accurately. • Curved interface boundaries approximated algebraically (by suitable basis functions) on regular rectangular or hexahedral grids, rather than geometrically on conforming meshes. • Small geometric details do not have to be resolved  coarser grids can be used. • Blending the best features of finite difference and finite element techniques. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

21. Related Computational Techniques • Heuristic homogenization by geometric averaging. • Homogenization based on variational principles. • Generalized FEM by Partition of Unity. • “Discontinuous Galerkin” methods. • “Finite Integration Techniques”. • Differential-algebraic treatment (chains and co-chains, the “discrete Hodge” operator in electromagnetic analysis, etc.). Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

22. Variational-Difference Schemes with Flexible Local Approximation • stencil(i) = {all M(i)nodes  (i)} • (i) = span{(i)},  = 1,2, …, m(i)  M(i) • uh(i) =  c(i)(i) • Test functionals {(i)}, supp((i))  (i) [u, u] = (f, u) [uh(i), (i)] = (f, (i)), i,  (i) (i) , uh(i)(i) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

23. The Nodal Basis m basis functions M nodes (m, M may depend on i ) Matrix of nodal values: For m = M,nodal,(r) = ; u(i)nodal = Nc (i) RM uh(i) = cT(i)= u(i)Tnodal NT(i) = u(i)Tnodal (i)nodal  (i)nodal = (N(i))T(i) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

24. The Variational-Difference Scheme Continuous and discrete problems: [u, u] = (f, u), u [uh(i), (i)] = (f, (i)), i,  (i) (i) , uh(i)(i) Diff. scheme: [ (i)nodal, (i)] = (N(i)) T [(i) , (i)] where [ , ] is applied entry-wise to the column matrix (i) For m  M,least squares: N(i)(N(i)TN(i))1 [(i) , (i)] Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

25. A Machine for VDS • n nodes • Cover:  = (i) • Basis functions {(i)} • Test functionals {(i)}, supp((i))  (i) i = 1,2, …, n Input N(i) (N(i)TN(i))1 [(i) , (i)] Coefficients of the difference scheme Output Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

26. Trefftz-FAME Schemes (“Flexible Approximation MEthod”) • stencil(i) = {all M(i)nodes  (i)} • (i) = span{(i)},  = 1,2, …, m(i)  M(i) • uh(i) =  c(i)(i) • Test functionals {(i)}, supp((i))  (i) Approximating functions (i) chosen to satisfy the underlying differential equation (e.g. Laplace). No test functionals needed! Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

27. The Trefftz-FAME Scheme (continued) m basis functions M nodes (m, M may depend on i ) Matrix of nodal values: u(i)nodal = Nc(i); sT(i)u(i)nodal=0 c(i)  s(i) Null (N(i)T) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

28. A Machine for Trefftz-FAME • n nodes • Cover:  = (i) • Basis functions {(i)}satisfying the underlying differential equation. i = 1,2, …, n Input s(i) Null (N(i)T) Coefficients of the difference scheme Output Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

29. Example: Variational-Difference Scheme; Basis Functions by Spatial Mapping  u = ;  – piecewise-constant. Machine input: stencil= 7-pt (3D); (i) = span{1, udipole_x, udipole_y, udipole_z, x~2, y~2, z~2}; (i) = “rectangular window” [flux balance]. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

30. Potential Distribution for Circular Cylinder: VDS vs. Exact Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

31. rparticle = 0.14 Errors: VDS vs. Standard Midpoint Scheme; circular cylinder standard scheme: control volume – flux balance, material parameter at midpoints Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

32. A Problem with Several Particles (FEMLAB rendition) plot along the dashed line on the next slide Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

33. A Problem with Several Particles (VDS vs. FEMLAB) potential vs. x at y = 0.3 solid line: FEMLAB (10489 nodes, 20736 2nd order triangles) squares: VDM; spatial mapping and dipole basis triangles: standard 5-point scheme VDM, mesh 10*10 Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

34. Errors: VDS vs. Standard Midpoint Scheme; ellipsoidal particle Basis functions correspond to “dipole” solutions in the vicinity of the particle Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

35. FD-Trefftz Examples Laplace eqn. Machine input: stencil= 5-pt (2D); (i) = span{1, x, y, x2  y2}. Machine output:standard 5-pt scheme. Laplace eqn. Machine input: stencil= 9-pt (2D); (i) = span{1, x, y, xy, x2  y2, x(x2  3y2), y(3x2  y2), (x2  y2) xy, (x2  2xy  y2)(x2 +2xy  y2)}; Machine output:a 6th order scheme (if hx = hy). Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

36. FD-Trefftz Examples (continued) Laplace eqn. Machine input: stencil= 19-pt (3D); (i) = {25 harmonic polynomials in x, y, z up to order 4}. Machine output: the 4th order ‘Mehrstellen’ scheme (L. Collatz). The 19-point Mehrstellen scheme for hx= hy= hz. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

37. Trefftz-FAME Examples (continued) Laplace or Poisson-Boltzmann eqn.around a particle.Machine input: stencil= 7-pt (3D); (i) = {spherical harmonics}; Machine output:a 7-pt scheme. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

38. Second Mini-Summary • Multivalued approximation & machines for difference schemes with arbitrary approximating functions. • Curved and slanted boundaries can be accurately represented on relatively coarse nonconforming grids. • Coulomb and Debye-Hückel potentials are also built into the scheme exactly. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

39. Some Simulations Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

40. Particles in Solvent:(1) Single polarized particle Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

41. (2) Two particles, FEMLAB simulation Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

42. (3) Two particles: Trefftz-FAME vs. Other Methods Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

43. Line Charge Near Slanted Boundary, Revisited  = 80  = 1 FEMLAB for verification (Mesh: 12505 nodes, 24768 2nd –order triangles.) Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

44. Line Charge Near Slanted Boundary: Trefftz-FAME, VDM and FEMLAB  = 80  = 1 Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

45. Third Mini-Summary • Flexible Approximaiton MEthods are quite promising in a large variety of applications, and often provide much higher accuracy at comparable computational cost. • Generalized FEM has been parallelized. FD-FAME scheems need further testing for large-scale simulations. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

46. Parallelization of the Generalized FEM Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

47. The PILUTS Solver (Achim Basermann) • NEC´s general-purpose suite of parallel iterative solvers (PILUTS: Parallel Incomplete LU with Threshold preconditioned Solvers). • Parallel sparse solvers for real symmetric positive definite (spd), general real symmetric and real non-symmetric matrices. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

48. The PILUTS Solver (continued) • Iterative methods: Conjugate Gradient (CG), symmetric Quasi-Minimal Residual (symQMR), Bi-Conjugate Gradient stabilized (BiCGstab) and Flexible Generalized Minimal RESidual (FGMRES). • Preconditioners: scaling, symmetric or non-symmetric incomplete block factorizations with threshold, and Distributed Schur Complement (DSC) algorithms. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

49. The PILUTS Solver:additional features • System graph can be re-partitioned by ParMETIS and re-distributed to reduce couplings between sub-domains and to accelerate convergence. • Matrix data can be re-ordered by METIS nested dissection in order to reduce the fill-in for local incomplete decompositions. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation

50. The NEC Linux Cluster GRISU • 32 2-way SMP nodes. • AMD Athlon MP 1900+ CPUs, 1.6 GHz. • 1 GB main memory per node. • Myrinet2000 interconnection network between the nodes. Achim Basermann, Igor Tsukerman, Parallel schemes with flexible approximation