Parallel Algorithm Design & Analysis Course Dr. Stephen V. Providence

1. Parallel Algorithm Design & Analysis CourseDr. Stephen V. Providence Motivation, Overview, Expectations, What’s next.

2. Motivation • The most important element of parallel computation and the primary focus of this course is the parallel algorithm, without which no problem could be solved in parallel. Dr. Stephen V. Providence Parallel Algorithms

3. Motivation • The manner in which algorithms occupy a central place in computer science, parallel algorithms occupy the center in parallel computation. • Given a problem to be solved in parallel, a parallel algorithm defines how the problem can be solved on the given parallel computer. (i.e. Grid Computing System) Dr. Stephen V. Providence Parallel Algorithms

4. Motivation • Specifically, parallel computation must specify how the problem is divided into sub-problems, how the processors communicate, and how the partial solutions are combined to produce the final answer. Dr. Stephen V. Providence Parallel Algorithms

5. Motivation • An understanding of the role of parallel algorithms and a good knowledge of the techniques of designing and analyzing them are essential to anyone entering the field of HPC or Computational Science. Dr. Stephen V. Providence Parallel Algorithms

6. Overview • To be taught in a manner analogous to a course on sequential algorithms, • this course will apply a similar course schema to parallel algorithm design and analysis. • there are topics unique to parallel algorithms. Dr. Stephen V. Providence Parallel Algorithms

7. Overview • Start with simple searching and sorting algorithms. • Then design them for parallel execution over given parallel topology (hypercube, mesh, grid), • details in the syllabus. • Progress made to more advanced algorithms, • such as: sorting networks, polynomial products, DFT and FFT. • details in the syllabus. Dr. Stephen V. Providence Parallel Algorithms

8. Overview • The textbook will outline the principles and give students and in-hand guide. • Handouts may be e-mailed to students along with technical data from journal articles. • The following texts are under consideration: • S.G. Akl, Parallel Computation: Models and Methods (reference). • S.G. Alk, The Design and Analysis of Parallel Algorithms (source) • M.J. Quinn, Parallel Programming in C with MPI and OpenMP (practical). Dr. Stephen V. Providence Parallel Algorithms

9. Overview • Programming in Parallel is an essential component of the course. • All algorithms must be coded and run on an available parallel computing system. • The binding language of choice is MPI. • We may use C or HPF (ForTran), probably C. • HPC++ and Java are possibilities (not Perl). Dr. Stephen V. Providence Parallel Algorithms

10. Overview • This course lends itself well to NCREN; • much of the work can be done online, • parallel codes easily demonstrated over NCREN, • lectures explaining algorithm functionality can be given over NCREN, • presentations of complex parallel computingconcepts can be given visually (animations), • exams and the like may be done off-line. Dr. Stephen V. Providence Parallel Algorithms

11. Overview • Culmination of course: • work on real research project, • execute algorithm design cycle, • preliminary algorithms designed and analyzed, • prototype codes created and tested, • codes verified and run, • extend and refine, • repeat design cycle. Dr. Stephen V. Providence Parallel Algorithms

12. Overview • Real world problem: • research in sparse matrix by vector products, • current research relies on matrix transformations, • use of structured matrices (Vandermonde, Cauchy, Toeplitz, Hankel, and matrices of related structure), • create diagonally dominant block matrices, • utilize super-fast linear system solvers, • all derived algorithms are highly parallelizable (sic), • applications to material science, CFD, and pure mathematics, • details in the syllabus. Dr. Stephen V. Providence Parallel Algorithms

13. Expectations • All students will appreciate the relevanceand utility of parallel algorithms in HPC; • their realization of parallel algorithm, importance will force them to seek the relevant algorithm first, then code the efficient solution (same as sequential). Dr. Stephen V. Providence Parallel Algorithms

14. Expectations • They will be able to solve problems more efficiently and know the limits of their solution (time and memory constraints); • NP-hard and NP-complete problems, • principle of parallel synergy (not limited by Amdahl’s speedup rule), • real time parallel computing (SETI@home). Dr. Stephen V. Providence Parallel Algorithms

15. Expectations • They will have the facility to extend well-known algorithms to solve specific problems; • many interesting problems require modification of parallel algorithm to exploit hidden parallelism, • often these modifications are minor. Dr. Stephen V. Providence Parallel Algorithms

16. Expectations • Focus is on algorithms: • not parallel architecture, • not parallel programming. • Although this course uses elements of both, it will give the student a foundation for work in HPC. Dr. Stephen V. Providence Parallel Algorithms

17. What’s next • Tasks: • set timetable • generate syllabus, • lecture schedule, • assignment, quiz and exam schedule, • grading rules, • choose textbook (listed above), • select computing platform (IBM p-series 690), • programming language (C or ForTran 90/95 w/ MPI, HPF, HPC++…). Dr. Stephen V. Providence Parallel Algorithms

18. What’s next • Contact:Dr. Stephen V. Providence http://www.comp.ncat.edu/faculty/providence svp@ncat.edu 336.334.7245 North Carolina A&T State University Department of Computer Science McNair Hall 503 1601 E. Market St. Greensboro, NC 27411 Dr. Stephen V. Providence Parallel Algorithms