Motivation

1. Transformation Issues and Implementation Motivation Main Transformation Issues Reduction Translation • The growth of data-intensive computing has been tied to the popularity of new programming paradigms: • Map-Reduce and some similar systems came up; • Various high level High Performance Computing Languages are developed. • Questions: I. Are HPC languages suitable for expressing data-intensive computations? • II. a) if so, what are the issues in using them? • II. b) if not, what characteristics of data-intensive computations force the need of separate languages? • Invoke the split function in FREERIDE; • Call reduction function to update the redobj; • Call combine ( and finalize) functions. • Main point is the two-stage mapping algorithm: • Collect the necessary information; • de-linearize the data set. Linearization Algorithm Two-stage algorithm, including: I. Compute the linearized data size; II. Linearize the data set. Data structure before and after linearization Chapel & Reduction Support data[l]: … data[l-1] data[1] data[0] … … b1[1] b1[0] b2 b2 b1[1] … b1[0] b1[n-1] b1[n-1] • Chapel supports two kinds of reduction model: • Local-view abstraction: • Straight-forward for implement; • Global-view abstraction: • accumulate: local reduction; • combine: global reduction; • generate: post-processing. … … a1[0] a1[1] a1[0] a1[1] … a2 a2 a1[m-1] a1[m-1] Linearizing Alg Mapping Alg Linear_data[ ]: b2 b2 … … b2 a1[0] a2 a1[0] a2 … … … a1[m-1] a1[m-1] m n l Experimental Results Conclusions • Present a case study for the possible use of a new HPC language for data-intensive computations; • Show how to transform the reduction features of Chapel down to FREERIDE middleware; • Combine the productivity of high level language with the performance of a specialized runtime system. Configuration PCA • CPU: Intel Xeon E5345 • 2 quad-core 2.33GHz • Memory: 6GB • OS: 64-bit Linux FREERIDE Middleware • Local Reduction • API function void (*reduction_t)(reduction_ags_t* ) is called, and the local reduction object is updated; • Global Reduction • API function void (*combination_t)(void* ) is called, and copies of reduction object are combined. K-means row=1000; column=10,000 Size=1.2G; k=10; i=10 row=1000; column=100,000 Size=1.2G; k=100; i=1 Size=12M; k=100; i=10 Acknowledgments: This work is supported by NSF awards to OSU and DARPA to Cray Inc