Parallel and Distributed Computing Early ( a nd Often)

1. Parallel and Distributed Computing Early(and Often) First NSF/TCPP Workshop on Parallel and Distributed Computing Education (EduPar-11)May 16th, 2011 Steven Bogaerts (speaker), Kyle Burke,Brian Shelburne Department of Mathematics & Computer Science Wittenberg UniversitySpringfield, OH Melissa Smith Department of Electrical & Computer Engineering Clemson UniversityClemson, SC Eric Stahlberg CCR Bioinformatics Core National Cancer InstituteBethesda, MD NSF grant CCF-0915805, SHF:Small:RUI:Collaborative Research: Accelerators to Applications – Supercharging the Undergraduate Computer Science CurriculumPIs: Eric Stahlberg, Melissa Smith, Steven Bogaerts

2. PDC Concepts Throughout the CS Curriculum Steven Bogaerts Algorithms Programming Languages Computational Modeling CS2 CS1 CS0 or Computer Literacy

3. Computer Literacy Steven Bogaerts Scratch

4. Computer Literacy Steven Bogaerts

5. Computer Literacy – PDC Overview Steven Bogaerts • Concepts • The world is “obviously” parallel. • Big-picture descriptions of some applications. • Terminology, history • Physical activities • Low-level: binary adder • Higher-level: card sorting • Communication • Shared memory vs. message passing

6. Computer Literacy – Curriculum Initiative Connections • Cross-Cutting • Why and what is PDC? • Concurrency topics : • Non-determinism • Power • Current/Hot/Advanced Topics • Cluster • Cloud/Grid • Algorithms : Algorithmic Problems : • Broadcast • Asynchrony • Programming : Semantics and Correctness Issues : Tasks and Threads • Architecture : Classes : Multicore Steven Bogaerts

7. CS1 Steven Bogaerts Parallelism/Concurrency can be used as a medium for teaching core computer science topics.

8. Analogy:Object-Oriented Programming Steven Bogaerts • Years ago: • OOP was a new paradigm • Should OOP be taught in introductory computer science courses? • Should OOP topics be “sprinkled” into existing courses? • If so, which topics should be taken out to make room? • Now: • Not just an advanced OOP course • CS1 • Data structures • Graphics • … • Topics taught through the medium of OO

9. Goals(CS1 and Beyond) Steven Bogaerts To prepare students for the “impending ordinariness” (Doug Lea) of parallelism and concurrency. Minimize the amount of material that must be removed to “make room.” Prevent the necessity of adding and staffing another course.

10. CS1 Steven Bogaerts • Introduction similar to Literacy course • Terminology, high-level concepts, physical activities • Basic use of Python multiprocessing module • Flexible application to many kinds of problems • Minimal additional syntax

11. Python Multiprocessing Module Steven Bogaerts

12. Python Multiprocessing Module Steven Bogaerts lockVar = Lock()lockVar.acquire()…lockVar.release() procVar.join()

13. Parallelism/Concurrency as a Medium Steven Bogaerts • Classes and objects • Parameter passing • Modularity and abstraction • “Sandbox” for exploration of other concepts: • Computing terms of the quadratic formula • Searching or sorting a list • Pattern matching • Clustering • Simulations of parallel processes in nature • … • Key idea: “Do what you usually do, just do (some of) it in parallel.”

14. Computational Modeling –Motivations Computational models are increasing in complexity Data available for constructing computational models is rapidly growing Parallel computing platforms are readily available Extreme Scale computing (exascale) platforms require new algorithms built from the ground up using parallel computing concepts Industry needs better performing models and parallel and accelerated computing are the only real options

15. Computational Modeling –Enhancement Objectives • Primary • Provide examples of easy parallel computing for models (embarrassingly parallel) • Provide introduction to use of 2D arrays in parallel computing environments • Keep it straightforward and accessible • Secondary • Provide introduction to contention and overhead

16. Computational Modeling – Approach • Use Computational Models and Methods course • 200 level (sophomore level) • Pre-requisite: Calculus, CS1 • Implementation using Mathematica Parallel Computing Toolkit • Mathematica is proven effective for computational models and methods • Functional programming • Ready with built-in parallel programming primitives starting with version 7 • Machine-independent • Select examples and provide a lab experience using primitives

17. Computational Modeling – Trial Lessons Learned • Students can master easy primitives • Students can even become comfortable with parallelizing matrix access by index • Mathematica more difficult for high-level parallelism • “Empty Kernel” starting point requires significant distribution of data and functions to start parallelism

18. Computational Modeling – Simple Parallel Examples • Data reduction (simple) • Sum[a,b,c,…] <-> ParallelSum[a,b,c,…] • Product[a,b,c,…] <-> ParallelProduct[a,b,c,…] • Matrices (data sharing) • Shared[{a,b,c}] • Other new fun options to try • ParallelPlot3D[] • ParallelDensityPlot[] • ParallelContourPlot[]

19. Programming Language Courses Steven Bogaerts • Exploration of programming paradigms • Include HPC languages! • Language: • Supports multi-threading • Alleviates common concurrency issues • Simplifies vital parallel patterns

20. Computer Organization Courses Steven Bogaerts • Already implicitly include parallelism! • Combinatorial circuits • Pipelines, super-scalar architectures • I/O with interrupts and DMA • Make the inherent parallelism more explicit • Consider VHDL • Example: • Carry look-ahead adder (parallel propagation of carrys) vs. ripple carry adder

21. Conclusion Steven Bogaerts • Early work in applying parallelism/concurrency as a medium for learning computer science. • Developments in: • CS1 • Programming languages • Computer organization